Structural Inference with Dynamics Encoding and Partial Correlation Coefficients
Authors: Aoran Wang
Jun Pang
Venue: ICLR 2024 poster (2024)
Keywords: Structural Inference
AI4Science
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)

Abstract: This paper introduces a novel approach to structural inference, combining a variational dynamics encoder with partial correlation coefficients. 
In contrast to prior methods, our approach leverages variational inference to encode node dynamics within latent variables, and structural reconstruction relies on the calculation of partial correlation coefficients derived from these latent variables.
This unique design endows our method with scalability and extends its applicability to both one-dimensional and multi-dimensional feature spaces.
Furthermore, by reorganizing latent variables according to temporal steps, our approach can effectively reconstruct directed graph structures. 
We validate our method through extensive experimentation on twenty datasets from a benchmark dataset and biological networks. 
Our results showcase the superior scalability, accuracy, and versatility of our proposed approach compared to existing methods.
Moreover, experiments conducted on noisy data affirm the robustness of our method.

Published as a conference paper at ICLR 2024
STRUCTURAL INFERENCE WITH DYNAMICS ENCOD- ING AND PARTIAL CORRELATION COEFFICIENTS
Aoran Wang's & Jun Pang 1.2
Faculty of Science, Technology and Medicine, University of Luxembourg
2 Institute for Advanced Studies, University of Luxembourg faoran. wang, jun.
ABSTRACT
This paper introduces novel approach to structural inference, combining a vari- ational dynamics encoder with partial correlation coefficients. In contrast prior methods, our approach leverages variational inference to encode node dynamics within latent variables, and structural reconstruction relies on the calculation of partial correlation coefficients derived from these latent variables. This unique design endows our method with scalability and extends its applicability to both one-dimensional and multi-dimensional feature spaces. Furthermore, by reorga- nizing latent variables according to temporal steps, our approach can effectively reconstruct directed graph structures. We validate our method through extensive experimentation on twenty datasets from a benchmark dataset and biological net- works. Our results showcase the superior scalability, accuracy, and versatility of our proposed approach compared to existing methods. Moreover, experiments conducted noisy data affirm the robustness of method.
INTRODUCTION
Complex dynamical systems often find their mathematical representations as networks of interact- ing agents. In this conceptualization, the system can be visualized as an interaction graph, where individual agents are nodes, their interactions are edges, and the underlying system's 's structure en- capsulated within an adjacency matrix. This modeling paradigm transcends diverse domains, find- ing applications in understanding physical systems (K (Kwapieñ Drožd7 2012 Ha & Jeong [021); unraveling multi-agent dynamics (Brasó & Taixe tricate interplay within biological systems (Tsubaki al.] 2 of understanding dynamical systems gains momentum, there arises an imperative-unvelie structure concealed within the intricate web of interactions. The recognition of this hidden frame- work not only advances comprehension of the system's 's intrinsic mechanisms but also empowers us to predict and control its behavior, goal of paramount significance. However, a common chal- lenge prevails across these scenarios: the availability of only fraction of nodes' observable features within a limited temporal window. This limitation leaves the underlying structural skeleton partially or completely veiled by the complex dynamical processes at play. To surmount this challenge, need arises for an approach capable of reconstructing the concealed structure of dynamical systems solely from the observable features.
Li et al. 2022), )) and deciphering the in- al. (020). As the pursuit
the
We introduce the notion of a trajectory, which amalgamates observed features from all nodes over defined time interval. These trajectories encapsulate rich tapestry information, encoding the evolution of node features a result of their historical states and interactions with fellow agents. Several methodologies grounded in variational autoencoders (VAEs) have emerged with the aim of unearthing the underlying structure based on these trajectories (Kipf et al.) 2018 Alet et al.) 2019) Webb et al. 2019 Chen et al.) 2021 Löwe et al. 2022 Wang Pang 2022 Wang et al.) 2023a). Guided by the Information Bottleneck (IB) principle, they navigate the reconstruction of dynami- cal systems' adjacency matrices within their latent spaces. Yet, a substantial limitation looms large: these approaches mandate global perspective of the entire system, rendering structural inference large-scale graphs computationally infeasible, In contrast, correlations and partial correlations offer an alternative route structure recovery, demonstrating scalability irrespective of graph size (Whit-

---

Published as a conference paper at ICLR 2024
taker 1990 Pratapa al. 2020). However, they require the node features to be one-dimensional every time step and lack the versatility to decipher directed graph structures.
In response to these challenges, we introduce novel structural inference framework that harnesses the strengths VAEs and partial correlations, Structural Inference with Dynamics Encoding and Partial Correlation Coefficients (SIDEC). Leveraging a VAE variant known as the variational dy- namics encoder (VDE) (Hernández et al.] 2018), we encode node feature dynamics over a de- fined temporal window into latent variables. Concurrently, we reduce feature dimensionality by extracting essential information-creanl tent variables form the basis for calculating partial correlation coefficients (PCOR), enabling the reconstruction of underlying interaction graph structures. SIDEC transcends the limitations of its predecessors, enabling structural inference on extensive graphs without an exponential surge in com- putational demands. It extends its applicability to trajectories featuring both one-dimensional and multi-dimensional node features. Remarkably, through the reorganization of latent variables, tem- poral order information is extracted, enabling precise inference of directed graph structures. Exper- imental results validate the prowess of SIDEC, surpassing previous methods both accuracy and scalability, while demonstrating resilience to diverse noise conditions.
a versatile foundation for structural inference. These la-
Our contributions encompass the following aspects:
We propose novel structural inference approach, which is rooted in variational dynamics encoder and partial correlation coefficients.
This method effectively addresses scalability concerns, does not need a prior for structure, and provides versatile solution for structural inference across various feature dimensions.
Through innovative latent variable manipulation, we expand its capabilities to accurately infer directed graph structures.
Empirical validation underscores its superiority existing benchmarks and its robustness in the presence of additive Gaussian noise.
RELATED WORK
Structural inference with VAEs. Structural inference seeks to faithfully reconstruct the interaction graph underlying a dynamical system, relying solely on observational data representing the state of agents over time. Neural Relational Inference (NRI) (Kipf et al.) 2018) is a seminal work that ven- tured into structural inference, employing VAEs within fixed fully connected graph framework. Building on the foundation laid by NRI, subsequent research has expanded the horizons of this field. These extensions encompass the adaptation to multi-interaction systems (Webb et al. 2019). the incorporation of efficient message-passing mechanisms (Chen al. 2222), the exploration of modular meta-learning (Alet et al., 2019), the iterative elimination of indirect connections (Wang & Pang, 2022). and the development of efficient structural inference techniques integrated with reservoir computing networks (Wang et al.) 2023a). Despite these advancements, a common limi- tation among these methods is their reliance latent spaces to infer entire graph structures, which constrains their scalability, particularly when confronted with larger graphs. Moreover, their loss functions naturally introduce a uniform prior on the structure, which is not versatile for all graphs.
Structural inference with correlations and partial correlations. In addition to VAE-based struc- tural inference approaches, researchers have harnessed correlation and partial correlation method- ologies to unravel the structures of interaction graphs. These techniques encompass diverse strate- gies, including Pearson correlation (Maucher al.] 2011 Specht & Li 2017). distance correla- tion (Liu et al.) 202I), partial correlations (Papili Gao et al. 2018 Millington & Niranjan 2019). low-order partial correlations (Zuo et al.) 2014), and camippartial correlation coefficients (Kim) 2015). While these methods excel within specific domains, such gene regulatory networks and financial networks, they grapple with limitations. Notably, they struggle to accommodate multi- dimensional node features and lack the capability to discern the directionality of edges. To address these challenges, this work introduces a fresh perspective by repurposing VAEs to shift their focus from structural inference to encoding node dynamics with the help of VDE. By integrating partial correlation calculations into the reconstruction process, this approach overcomes scalability chal- lenges and ensures robustness across various node feature dimensionalities.
2

---

Published as a conference paper at ICLR 2024
3 PRELIMINARIES
In this section, we provide preliminaries and background knowledge.
Notations and problem formulation. We conceptualize dynamical system as directed underly- ing interaction graph, where the system's 's agents correspond to nodes, and the directed interactions among these agents manifest as edges in the graph. Denoted as G = (V.E), this directed graph comprises V, the feature set of n nodes, and and e, the set of edges. The temporal evolution of nodes' features is encapsulated in trajectories: V (V) (Vo, V1 ....,,T-11] VT-1J, spanning T time steps, with V signifying the feature set of all n. nodes at time step t: V (v)- uf...., - The feature vector at time for node i, denoted as o/ E R S S T - 1, is d-dimensional. Usually, a set of M trajectories is observed: [V[n). Vp....... V mn] ..... VIM}}. We assume that the nodes are observed in their entirety, and & remains immutable during the observation. From E, we derive an asymmetric adjacency matrix denoted as A E RR Within A, each element aij E (0. 1} indi- cates the presence (aij 1) or absence (a:j 0) of an directed edge from node to node j. The dynamics of the system are significantly influenced by A, with node i's features at time + 1 as:
of+1 0 + Σ ((lee,))laa))
(1)
jEli
where A denotes time interval, U is the set of nodes connected with node and f()) is the state- transition function deriving to dynamics caused by the edge from node j to i, and ||---|lca denotes the a-distance. Importantly, U_i is derived from the adjacency matrix A, highlighting the role of A in determining the interactions between nodes and the system's 's dynamics. This paper primarily fo- cuses on the challenge of structural inference, which involves the unsupervised reconstruction of the structure encapsulating the underlying interaction graph based on M observable sets of trajectories.
Information bottleneck. The Information Bottleneck (IB) theory has emerged as a valuable frame- work for understanding the inner workings of deep neural networks (Tishby et al. 1999 (Tishby & Zaslavsky 2015 Shwartz-Ziis & Tishby (017). At its core, IB aims to discover minimal sufficient representation, denoted as Z, for a given input data X and its label Y. This discovery is achieved by minimizing the expression [Z; X) - u I(;; Y). where u serves as a Lagrangian multiplier that balances the trade-off between sufficiency and minimality. Furthermore, the Variational Information Bottleneck (VIB) (Alemi et al. 2017). introduces a variational approximation to the IB theory. It unveils that the objective function of VAEs constitutes specific instance of this approximation. In the realm of unsupervised learning, VAEs excel at extracting the minimal sufficient statistics from input features, facilitating the derivation of output features while retaining compressed and abstract representations within their latent spaces.
Previously, VAE-based structural inference techniques (Kipf al. 2018 Webb al. et al. 2021 Wang & Pang 2022] Wang et al., (202a) have employed the VIB framework to recon- struct the adjacency matrix within the VAE's latent space. expressed as:
2019
Chen
Z = arg min [Zz A) u I(Z V*+1).
(2)
z
Initially, these methods assume fully connected graph A and subsequently sample À from the latent space Z. However, scalability issues arise when confronted with graphs exceeding 100 nodes, as they require learning representations for each node pair, necessitating processing of the entire graph. Additionally, the assumption of a uniform distribution in the Kullback-Leibler (KL) term of the loss function implies equal edge existence probabilities, an assumption unsuitable for all graphs. In contrast, our approach, SIDEC, harnesses the power of a Variational Dynamics Encoder (VDE) (Hernández et al., 2018). It conceptualizes the minimal sufficient statisties within the la- tent space as dynamics within the node features, offering both scalability and the elimination of assumptions regarding edge existence probabilities.
Partial correlations. Partial comelatiin is statistical concept that quantifies the association be- tween two random variables after removing the influence of all other random variables (Kim) 2015). In the context of this research, it serves to estimate the direct relationship or associa- tion between two nodes, essentially indicating the existence of interaction between them. Let's consider random vector X (21, 22, ... "..,,,,.,,,, where |X] = We denote the vari- ance of a variable random z; as Var;(= var(z,)) and the covariance between two random vari- ables I; and x, as Coviy(= x.), The variance-covariance matrices of random vectors
3

---

Published as a conference paper at ICLR 2024
Overview the Whole Pipeline
Variational Dynamics Encoder (VDE)
Encoder
Decoder
Calculation Partial Correlation Confficients
Calculation Partial Correlation Confiicient
Nith up.1 MASA Sq
(2,)
Figure I:
The pipeline of SIDEC. The wrapped box on the bottom left shows the pipeline of the calculation of partial correlation coefficients (PCOR).
X and Xs (S C 21, n and |S| < n) are denoted as Cx and Cs. respectively, where Xs is random sub-vector of X. The correlation between two random variables aa and x, is Cooss (= corr(xi, 2z)). The partial correlation Tijkk of x; with zz given ay is: Tij /VarsvVary
Tij
FakTik
Tayle
jk
For cases involving more than three variables, such as more than three nodes in graph, we extend the formulation to higher-order partial correlations. This involves considering the inverse variance- covariance matrix of X, denoted as Dx = Cx² Here, du and Cjj are the (i,h) elements of the matrices Dx and Cx, respectively. Then following (Whittaker 1990), the partial correlation of s; with zy given a random vector Xg is:
dij
TaJs
ydu / dis
X is the random sub-vector of X after removing the random variables x; and æj.. Its size is |S] [X] 2. For more details about the derivation, please refer to Appendix A While Eqn. provides
insights into the graph's 's structure, should be noted that partial correlation alone is insufficient. The methodology for obtaining the graph's ' structure is discussed in Section 4.3
4 METHOD
4.1
ENCODING DYNAMICS WITH VARIATIONAL DYNAMICS ENCODER
Unlike previous VAE-hased structural inference methods Kipf et al. 2018 Webb et al. 2019 Chen
et al. 2021 Wang & Pang 2022 Wang et al. 2023a), which employ VAES simultaneously learn edge representations and reconstruct the adjacency matrix of the graph in their latent spaces, SIDEC takes different approach. leverages a Variational Dynamics Encoder (VDE) (Hernández al. 2018) designed to capture minimal information about the dynamics of node features within defined time window. While the architecture of VDE bears some resemblance to a VAE, time-series data. The VDE is presented in the top row in Fig. [ Specifically, VDE predicts the state of a node (u!+7) at future time step + 7) based on its current state (uf), where T represents the length of the time window required for the system' 's dynamics to be considered Markovian.
tailored for
To clarify, prior VAE-based methods aim to preserve minimal sufficient statistics the latent space and formulate it as the adjacency matrix Á the graph, as depicted in Eqn. 2 In contrast, SIDEC employs VDE to learn minimal sufficient statistics essential for encoding node feature dynamics within a specific time window, which can be expressed through the VIB principle:
arg min / (z); uf) 2
[2];;
where of is the feature of node i at time t, and u is its feature after time window The information about the dynamics is encoded in the latent space of VDE.
Implementation. Suppose have set of M observable trajectories of a dynamical system with n nodes, with each trajectory covering the evolution of all node features within a time period T. At each time step, VDE processes the features of single node. VDE comprises three operations:
4

---

Published as a conference paper at ICLR 2024
encoder parameterized by ø that computes qo(z)oof), decoder parameterized by 0 that computes pa(ut" +7 (2]). and reparametrization step. The implementation of VDE can be formulated as:
Encoder:
1 of = ENC(O)).
(6)
Reparametrization:
z μ4 + of e where 1),
(7)
Decoder:
oitt DEC(2)),
(8)
where of the input node feature time t, o, the prediction of node feature after time window т. ENC and DEC are encoder and decoder operations, respectively, and are implemented as multi- layer perceptrons (MLPs). For the exact implementation of the encoder and decoder please refer to Appendix A.7 i and of are the outputs of the encoder, with @ as the element-wise production. To reduce the dimensionality for PCOR calculations, the scale of z to be one. Importantly, 2 = u/, ensuring that the learned variable & encodes the dynamics for node features from time to 1 +
Ad Advantages. Leveraging VDE to encode node dynamics offers several advantages:
1. Focus on dynamics:
By adhering to the VIB principle, VDE concentrates on capturing minimal sufficient statistics that encode the dynamics of node features, making particularly well-suited for modeling dynamical systems.
Scalability: Unlike previous VAE-based methods, VDE operates solely on node features and doesn't necessitate a complete view of the entire n-node system, enhancing scalability.
3. No uniform prior: VDE doesn't impose a uniform prior on the edges, unlike previous methods, making it more adaptable various graph types.
4. Dimensionality reduction: VDE can effectively reduce the dimensions of node features, ac- commodating dynamical systems with varying feature counts and enabling the use of correlation calculations, which inherently require one-dimension features.
4.2 TRAIN WITH HYBRID LOss
In this study, the training of the VDE hinges on composite loss function comprising two key components: the reconstruction loss, denoted as Creons and the autocorrelation loss, represented as Lantee The reconstruction loss, Lrecon, derives inspiration from the widely employed Evidence Lower Bound loss for VAEs (Kingma & Welling (001)). It comprises two integral segments: one that evaluates the prediction accuracy of node features after time window т, and another that assesses the KL divergence between latent space priors. Formally expressed as:
Lrcoon (x) +o(u*+7|2) DKL (Dz( po(z)).
(9)
This loss function is implemented as:
Crecon 1 Σ
|| 60t7 01+r122- 2L
02
(10)
where L signifies the batch size. The autocorrelation loss, Lantoes serves the purpose of opti- mizing VDE to capture comprehensive representations of long-time kinetics within time-series data (Hernández et al., 2018). This loss seeks to ensure that no process in the data evolves more slowly than the true underlying process, as proposed by Noé & Nuske (2013). A favorable outcome of this optimization process results in phase-space decomposition that yields a linear model with larger leading dynamical eigenvalues, enhancing the overall representation quality. For single- layer decomposition of phase-space, equivalent to the autocorrelation of μ (Noé & Nuske 2013) the autocorrelation loss is defined as:
E[(2" (1)) (T(+T)
Lautoe
(11)
Sprt
where ji and Sμ are the sample mean and population mean of the latent space vector for specific data batch. Consequently, the combined loss for training VDE formulated as:
LVDE
Lieon + Lautoc-
(12)
4.3 STRUCTURAL INFERENCE WITH PARTIAL CORRELATION COEFFICIENTS
The role of VDE extends beyond encoding node feature dynamics; it also encompasses the critical task of reconstructing the underlying graph structure based on this encoding. To begin, we collect the obtained & for all nodes, forming M sets of features denoted as Z [mm] 1 S m M), where
5

---

Published as a conference paper at ICLR 2024
[m)] E R(T-7)xn, For each [m]] employ Eqn. 4 to compute matrix of partial correlations
R. where the (i.j)-th cell of the matrix denotes the partial correlation of z; with zj given a random vector Xs. Tajss- Subsequently, the obtained R is utilized to calculate PCOR, including statistics
Lyjs and p-values Pijis, as detailed in (Weatherbum 1949: Sheskin
2003
PUjls = - [ti]ss]
2 --m) n) where tijss = rigls
(13)
1- 1-rijss
r
H is the sample size (= T - - for a Zm). n is the number of nodes in the graph. .,(-) signifies the cumulative density function of Student's t distribution with degrees of freedom H - The resultant structure of the graph is represented as P. with each (i, cell equals to Pujs- Across M trajectories, collection of M P matrices is obtained. However, in this symmetric configuration, the derived structure cannot capture the intricacies of directed graphs, which are characterized by unsymmetrical structures. To address this, we propose novel algorithm that leverages temporal information to infer directed edges.
From temporal order directed edges. In the real-world dynamics of systems, events typi- cally occur in chronological order. Building upon this observation, we utilize temporal infor- mation to infer the structure of a directed graph. We reorganize the M sets of features Z as Z [Zmm], 1 S m M). with each [m] (Z E IR
S S n). Each row of [Z(m],s) comprises [z) _ ..... z -1 combining the dynamics encoding of time step t for node with the dynamics encoding of time step t for the remaining nodes. Moreover, follow- }. This results in a matrix of p-values, denoted as P., estimating the directed edges from other nodes node i. We then compose the overall P as follows: P(:; = (, i). yielding an unsymmetrical matrix that effectively estimates the structure of a directed graph. SIDEC adopts this pipeline reconstruct the structure of directed graphs, i.c., the adjacency matrix A = P. For implementation details, please refer to Appendix A.7
ing Eqn. 13 we calculate p-values for each (Zmm],))
Advantages. Employing PCOR for structural inference offers several distinct advantages:
1. Enhanced accuracy: Benchmark assessments for structural inference have demonstrated that PCOR exhibit superior accuracy (Anonymous) 2023). The utilization these coefficients en- sures high precision in the reconstructed structure.
2. Dimensionality reduction: VDE initially reduces the dimensionality of the trajectories, broad- ening the applicability of PCOR to multi-dimensional trajectories.
Directed graphs: Incorporating temporal information allows the algorithm to infer directed graph structures, overcoming the limitation of symmetric structures.
5 EXPERIMENTS
In our evaluation, we subjected SIDEC to rigorous testing across a diverse range of 20 datasets. These datasets encompassed trajectories featuring both multi-dimensional and one-dimensional fea- tures. Our evaluation included robustness testing against trajectories with added Gaussian noise and an in-depth ablation study assessing the influence of time window selection.
5.1 GENERAL SETTINGS
The benchmark we employed for evaluation, StructInfer (Anonymous) 2023), provided a comprehensive set of datasets specifically tailored for structural inference tasks. We selected datasets related "Vascular Networks". which involved both Springs and NetSims dynamical simulations. These datasets comprised trajectories with varying numbers of nodes, denoted as E (15,30, 30, 50, 100, 150, 200, 250]. Each dataset was identified based on the type of dynamical simulation and the number of nodes. For instance, "VN_SP_15" denotes trajectories generated using Springs simulation with 15 nodes, while "VN.NS.3" NS_30" indicates trajectories generated using Net- Sims simulation with 30 nodes. Notably, Springs-generatee features at each time step. whereas NetSims NetSiim--ennrated trajectories featured mieddmensionnl fea- tures. This diverse dataset selection allowed us to assess SIDEC's performance across various fea- ture types and its capacity to conduct structural inference on larger graphs. Our data split followed the predefined training, validation, and test sets within the StructInfer benchmark.
Datasets.
trajectories possessed four-dimensional

---

Published as a conference paper at ICLR 2024
Table 1: Average AUROC Results (%) of SIDEC and baselines StructInfer datasets.
Methods
VN SP.15
VN SP_30 30
VN SP
VN SP. 100
VN SP.150
VN SP 200
VN SP 250
NRI
94.5= oa1
95.1 + 0.01
94.6+ 0.22
89. 00 002
OOM
OOM
OOM
MPM
96.6 0.01
89.7 0.04
85.1 0.2
84.6 003
OOM
OOM
OOM
ACD
94.3+ oa1
93.7+ 0,01
87.5+ 0.3
90.5 : 003
OOM
OOM
QOM
iSIDG
96.6+ 0.2
95.6+ 0.01
95.7+ 0,22
85.1 +.1++000 0.22
OOM
OOM
OOM
RCSI
97.01 0.01
95.31 0,01
94,51 0,02
90.7.1 003
OOM
OOM
OOM
SIDEC
97.6 0.01
97.0.1. 0,82
96.5 0,83
95.7.1 0002
95.31 0.82
95,5 0/12
95.0.t 0J3
Methods
VN_NS_15
VN_NS_50 VN_NS_30
VN_NS_100
VN_NS_150
VN_NS_200
VN_NS_250
NRI
90.31 0.01
74.6 0.04
7 0,00
68 08.8% 002
60.5
OOM
OOM
MPM
91.2= 0.01
83.4 0.0a
72.7+t
70.3+
63.5s
OOM
OOM
ACD
80.3 0.02
65.4 0.06
69. 1 n.00
68.7 0.3
62.8s
OOM
OOM
iSIDG
91.2 0.00
78. 0.06
73.7
68.8 0.2
64.1
OOM
OOM
RCSI
91.5+ 0.02
82.3+ 0.04
74.1
70.3+ 0.03
64.5+ 0.03
OOM
OOM
SIDEC
96.1# 0.03
97.9 0.81
98.5+
98.5 0.01
98.1+ 0.83
98.5 0.02
98.7 0m3
In addition to the StructInfer datasets, similar to previous works [Wang & Pang 2022 Wang et al.
2023a) we conducted evaluations on six directed synthetic biological networks (Pratapa al. 2020
with the names Linear (LI), Linear Long (LL.,, Cycle (CY), Bifurcating (BF), Trifurcating (TF), and Bifurcating Converging (BF-CV). These networks represented critical components involved in the differentiation and development of cells (Saelens et al. 2019). To simulate these networks, we used BoolODE (Pratapa et al.) (020), recording trajectories consisting 49 time steps. Subsequently, we performed subsampling of time steps these raw trajectories. Trajectories were randomly divided into training, validation, and test sets in ratio of 8:2:2. In this case, the features at each node represented mie-dimensional mRNA expression levels.
Baselines and metries. We compare SIDEC with the following state-of-the-arr VAE-based methods for structural inference directed graphs:
NRI (Kipf et al. 2018): : a VAE-based model for unsupervised relational inference.
MPM (Chen et al.) 2021): an NRI-based method with relation interaction mechanism and
spatio-temporal message passing mechanism.
ACD (Löwe et al. 2022): model that leverages shared dynamics infer causal relations.
iSIDG (Wang & Pangl 2022): a VAE-based model that iteratively updates the adjacency matrix to be fed to the encoder with direction information.
RCSI (Wang et al. 2023a): a reservoir computing network is integrated with a VAE for more efficient structural inference.
To evaluate and compare these methods, we utilized the area under the receiver operating char- acteristic (AUROC) metric assess the inferred adjacency matrix against ground truth data. For trajectories from StructInfer, every group of them is the results from three different graphs. For instance, "VN_SP_15" has three different graphs but with the same number of nodes. Following the instructions of the benchmark, we thus run methods on the one with the lowest performance, and then report the average AUROC results for all ten runs. Trajectories generated by synthetic biological networks underwent ten runs with varying random seeds, with the average AUROC results reported.
each graph three times, with extra
Experimental settings. All experiments 32G graphics card, paired with two Xeon Gold 6132 2.6GHz CPUs. The deep learning methods were trained for maximum of 600 epochs, with batch sizes, learning rates, and hyperparameters configured according to their respective original implementations. Further details are available in Appendix C while Appendices DDI D.2 provide more experimental results and comparison with other structural inference methods.
conducted single NVIDIA Tesla V100 SXM2
5.2 EXPERIMENTAL RESULTS
In this section, we delve into comprehensive analysis of the performance of all the methods applied to both VN trajectories from Structinfer and synthetic biological networks. The experimental out- comes of our proposed model and the baseline methods are concisely summarized in Tables and annd) Within Table I "OOM" designates "out-of-memory." indicating instances where certain methods couldn't be executed due memory limitations.

---

Published as a conference paper at ICLR 2024
Table Average AUROC Results (%) of SIDEC and baselines synthetic biological networks.
Methods
LI
LL
CY
BF
TF
BF-CV
NRI
70.5 nas
75,0+
64.5 0,85
59. 19.14. 000
55. 0.02
59.2+ + 0:4
MPM
75.0# uaz
79.2+
79.0= 0,8
63.5 002
58.44
64.4: 0.3
ACD
65.0# uas
68.4+ 0.2
62.9= 0.02
59.8 0.0
57.2 0.03
55.8 0.03
iSIDG
86.2 0.02
88, 0.02
79.5 0.02
68.3 0.02
60.2.t 0.03
70.7: 0.03
RCSI
88.5 0.03
91.01 0.03
81.0= 0.02
72.2 0.00
65.0:1 0.02
73.7 0.02
SIDEC
89.0 0,02
92.3
82.4
75.8 0,03
68.5
78.5 0.02
Table analysis shows clear trends. When confronted with trajectories featuring multi-dimensien feature like VN. SP, SIDEC outperforms VAE-based methods significantly in graphs with over 100 nodes, demonstrating remarkable scalability and proficiency structure reconstruction, a trend also evident in VN_NS trajectories. Contrasting with baseline sensitivity to node count changes. SIDEC consistently excels in reconstruction accuracy. Furthermore, in VN_NS experiments, while baselines appear sensitive to changes in graph node counts, SIDEC maintains consistently high performance in terms of structure reconstruction accuracy. This pattern of superiority is further affirmed by experiments six synthetic biological networks, with SIDEC's exceptional performance attributed to these key factors:
1. Utilization of VDE for dynamics encoding: Unlike conventional VAB-based baseline models, SIDEC leverages a VDE to encode the dynamics individual nodes into latent variables. This approach operates specifically
node features, avoiding the need to process the entire graph. This feature imparts exceptional scalability, especially for large graphs. Moreover, stands out by not imposing a uniform prior the inferred structure.
2. Synergy between PCOR and VDE: As evident from the StructInfer benchmark, the calculation of PCOR, when used in isolation, exhibits remarkable accuracy in reconstructing graph struc- tures. However, its applicability is constrained, as it cannot handle multi-dimensional features or infer structures in directed graphs. our proposed methods, VDE plays a dual role: it reduces feature dimensionality and encodes dynamics into the latent variable, which is subsequently uti- lized in the calculation of PCOR. This symbiotic relationship harnesses the strengths of both PCOR and VDE, resulting in significantly enhanced performance.
3. Introducing directionality through temporal ordering: SIDEC introduces directional edges by considering temporal ordering. This innovation substantially broadens the utility of PCOR for structural inference in directed graphs. As demonstrated in the tables, this extension seamlessly restores the accuracy and scalability of PCOR.
4. Versatility in edge priors: SIDEC does not rely a uniform prior for edges, making them exceptionally versatile. They can be applied across broader spectrum of graph types. The VN datasets, characterized by diverse properties, resist summarization via uniform prior. This is methods.
precisely where SIDEC outperforms
5.3 ROBUSTNESS TESTS
In this section, we delve into the perfor- mance degradation of SIDEC when ap- plied to VN trajectories from StructInfer, which have been subjected to various lev- els of added Gaussian noise. We utilize of datasets: VN_SP and VN_NS trajectories, each intentionally perturbed with five distinct levels of Gaussian noise from the StructInfer dataset. To quantify the performance differences, we calculate AAUROC AUROC AUROCNo- where AUROC signifies the average AUROC results for pristine, noise-free tra- jectories, and AUROC represents the Gaussian noise, X E 2,, 3,4,5, 5).. These
Springs
NotSims
-2
AATROC (in %%
-4
-6"
-10
Levels
415
250
350
3180
+150 +200
2230
Figure 2: Performance differences (in %) SIDEC on VN trajectories with Gaussian noise of different levels.
average AUROC values obtained under X level of added results are visually presented in Fig. 2 As evident from both figures within Fig, 2 the inclusion of noise in trajectories indeed exerts negative influence the performance of SIDEC. This perfor-
8

---

Published as a conference paper at ICLR 2024
mance degradation is linearly correlated with the intensity of noise. However, it is noteworthy that despite the noise, the performance drops remain below 10%. This observation underscores SIDEC's robustness against added Gaussian noise, ranging from moderate to high. Interestingly, the left side of Fig. 2 reveals that the performance drops for larger graphs are consistently smaller than those observed for smaller graphs. For instance, the line representing the performance drop on VN_SP_30 consistently surpasses that of VN_SP_15 .15, and similar trend is noted for other line pairs. In contrast, the right side of Fig. 2 presents less distinct pattern for VN NS trajectories, with lines exhibiting greater overlap. This intriguing observation suggests that SIDEC excels at extracting and encoding rich information from malli-dimensional features into the latent variable. When applied larger graphs. this results in more substantial amount of information encoded within the latent variable, enabling SIDEC to effectively mitigate the adverse effects of noisy features.
5.4
ABLATION STUDY ON THE LENGTH OF TIME WINDOW
In this section, we delve into the critical aspect of selecting the appropriate length for the time window, denoted as T, which has the potential to significantly impact the performance of SIDEC when applied to VN trajectories sourced from Struct- Infer. As mentioned in Section 4.1 T is chosen based on the dynamics between of and tf in manner that ensures the system's 's dynamics exhibit Markovian behavior. However, due to the continuous evolution of dynamical systems and the in- herent discrete nature of the obtained trajectories, our assess- ment of the system's 's Markovian properties is an approxima- tion. To pinpoint the most suitable value for the time window т, we conduct an ablation study using noise-free free VN_SP_15 and VN_NS.15 trajectories. Our test set for encompasses values drawn from the {1, 3, 4, 5, 6, 7,8, 9, 10), and the results are averaged over ten independent runs, presented in Fig. As evident from the figure, when т increases, the performance of SIDEC both VN_SP-15 and VN_NS-15 gradually deteriorates, with more pro- nounced performance drops observed at larger T values. This phenomenon stems from the fact that selecting an excessively large time window disrupts the underlying mechanism of VDE. When too many time steps are skipped, the node feature dynamics become excessively chaotic, rendering VDE less effective at capturing and encoding these dynamics. Consequently, based on the insights derived from this ablation study, we opt for 7 = as the most suitable choice.
100.0
VN SP 15
97.5
IS
AUROC (%)
95.0
92.5
90.0
87.5
85.0
10
Figure 3: Average AUROC values (in %) of SIDEC-T on noise-free VN SP 15 and VN NS 15 trajec- tories with different values of r.
6 CONCLUSION
In this paper, introduced innovative structural inference method, employing VDE in tandem with partial correlation coefficients. VDE serves a dual purpose: encoding node dynamics into latent variables and extracting essential information from node features reduce dimensionality. These latent variables enable precise graph reconstruction via partial correlation coefficients, accommo- dating trajectories with one-dimensional and multi-dimensional features. Notably, our approach sidesteps the need for a comprehensive graph view, granting it exceptional scalability for large graphs, a long-standing challenge. Through temporal reordering of latent variables, our method is able to reconstruct directed graph structures. Experimental results validate its efficacy across diverse datasets and large-scale graphs, showcasing robustness against Gaussian noise.
Currently, due to the formidable challenges involved in acquiring real-world data, our experimental validation is confined to synthetic data. A promising avenue for future research lies in the collection and curation of real-world datasets tailored for structural inference tasks. Furthermore, inspired by recent work (Wang et al., 2023b), the exploration of individualized local receptive fields for each node presents another intriguing avenue for future investigation. Such local biases, when incorpo- rated during structural inference, hold the potential to uncover deeper insights into complex real- world networks. Besides that, it would also be interesting develop structural inference method that can deal with the incomplete view of nodes, where some nodes are unobservable. These promis- ing research directions pave the way for advancements in structural inference.

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
To ensure both the reproducibility and comprehensiveness of this paper, we have included an ap- pendix containing detailed implementation information. Additionally, the code be found at Appendix [..] presents the pseudo-
https: //tp:///gtttbbbuu com/ wttp////nn2203//SECCCCCCCCCCCCC code outlining the PCOR method used in this work, while Appendix AA.6 offers the pseudo-code for the complete SIDEC pipeline. Furthermore, Appendix [A.7 furnishes a comprehensive list of instructions how to construct SIDEC from scratch. For thorough understanding of the datasets employed in our research, please refer to Appendix B where we provide detailed descriptions of their characteristics and sources. With these comprehensive resources, we have taken diligent steps to ensure the reproducibility of our work.
ACKNOWLEDGMENTS
Author Jun Pang acknowledges financial support from the Institute for Advanced Studies of the Uni- versity of Luxembourg through Audacity Grant (AUDACITY-201I) The experiments presented in this paper were carried out using the HPC facilities of the University of Luxembourg (Varrette) al. 2022) (see hpc. uni. lu).
REFERENCES
Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, and Kevin Murphy, Deep variational information bottleneck. In Proceedings of the 5th International Conference on Learning Representations (ICLR), 2017.
Ferran Alet, Erica Weng, Tomás Lozano- Pernno-éére,,.ee and Leslie Pack Kaelbling. Neural relational infer- ence with fast modular meta-learning. In Advances Neural Information Processing Systems 32 (NeurIPS), 2019.
Theodore Wilbur Anderson. An Introduction to Multivariate Statistical Analysis, volume 2. Wiley New York, 1958.
Authors Anonymous. Benchmarking structural inference methods for dynamical interacting sys- tems.
2023.
James Biagioni and Jakob Eriksson. Inferring road maps from global positioning system traces:
Survey and comparative evaluation. Transportation research record, 2291(1):61-7,, 2012.
Guillem Brasó and Laura Leal-Taixé. Learning a neural solver for multiple object tracking. In
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6247-6257, 2020.
Thalia E Chan, Michael PH Stumpf, and Ann C Babtie. Gene regulatory network inference from single-cell data using multivariate information measures. Cell Systems, 5(3):251-267, 2017.
Chao Chen, Karl Petty, Alexander Skabardonis, Pravin Varaiya, and Zhanfeng Jia. Freeway perfor- mance mcasurement system: mining loop detector data. Transportation Research Record, 1748 (1):96-102, 2001.
Siyuan Chen, Jiahai Wang, and Guoqing Li. Neural relational inference with efficient message pass- ing mechanisms. Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAT), pp. 7055-7063, 2021.
Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM International Conference on Knowledge Discovery and Data Mining (KDD), pp- 785-794. ACM, 2016.
P ERDdS and A R&wi. On random graphs i. Publ. math. debrecen, 6(290-297):18, 1959.
Jeremiah Faith, Boris Hayete, Joshua T Thaden, Ilaria Mogno, Jamey Wierzbowski, Guillaume Cottarel, Simon Kasif, James Collins, and Timothy S Gardner. Large-scale mapping and val- idation of Escherichia coli transcriptional regulation from a compendium of expression profiles.
PLoS Biology, 5(1)ee8, 2007.
10

---

Published as a conference paper at ICLR 2024
Seungwoong Ha and Hawoong Jeong. Unraveling hidden interactions in complex systems with deep learning. Scientific Reports, 11(1):1-13, 2021.
Cine-Claire Haury, Fantine Mordelet, Paola Vera-Licona, and Jean-Philippe Vert. TIGRESS: trust- ful inference of gene regulation using stability selection. BMC Systems Biology, 6(1): 2012.
Carlos X. Hernández, Hannah K. Wayment-Steele, Mohammad M. Sultan, Brooke E. Husic, and Vijay S. Pande. Variational encoding of complex dynamies. Physical Review E, 97(6):062412, 2018.
W. Huber, J. Carey, R. Gentleman, S. Anders, M. Carlson, B. S. Carvalho, H. C. Bravo, S. Davis, L. Gatto, T. Girke, R. Gottardo, F. Hahne, K. D. Hansen, R. A. Irizarry, M. Lawrence, M. Love, J. MacDonald, V. Obenchain, A. K. Ole's, H. Pag es, A. Reyes, P. Shannon, G. K. Smyth, D. Tenenbaum, L. Waldron, and M. Morgan. Orchestrating high-throughput genomic analysis with Bioconductor. Nature Methods, 12(2):115-121, 2015.
Vân Anh Huynh- Thu and Pierre Geurts. dynGENIE3: dynamical GENIE3 for the inference of gene networks from time series expression data. Scientific Reports, 8(1):3384, 2018.
Seongho Kim. ppcor: an R package for fast calculation to semi-partial correlation coefficients. Communications for Statistical Applications and Methods, 22(6):665, 2015.
Diederik P. Kingma and Max Welling. arXiv:1312.61 14, 2013.
Auto-encoding variational bayes. arXiv preprint
Thomas Kipf, Ethan Fetaya, Kuan-Chich Wang, Max Welling, and Richard Zemel. Neural rela- tional inference for interacting systems. In Proceedings of the 35th International Conference on Machine Learning (ICML), pp. 2688 2698-269. PMLR, 2018.
Jarostaw Kwapień and Stanislaw Droždz. Physical approach to complex systems. Physics Reports, 115(3):115-226, 2012.
Jiachen Li, Hengbo Ma, Zhihao Zhang, Jinning Li, and Masayoshi Tomizuka. Spatio-temporal graph dual-attention network for multi-agent prediction and tracking. IEEE Transactions on Intelligent Transportation Systems, 23(8): 10556- 10559,, 2022.
Kuan Liu, Haiyuan Liu, Dongyan Sun, and Lei Zhang. Network inference from gene expression data with distance correlation and network topology centrality. Algorithms, 14(2), 2021.
Sindy Löwe, David Madras, Richard Z.. Shilling, and Max Welling. Amortized causal discovery: Learning infer causal graphs from time-series data. In Proceedings of the 1st Conference Causal Learning and Reasoning (CLeaR), pp. 509-525. PMLR, 2022.
Baoshan Ma. Mingkun Fang, and Xiangtian Jiao. Inference of gene regulatory networks based on nonlinear ordinary differential equations. Bioinformatics, 36( 49):18944884488 4893, 2020.
Adam A Margolin, Iya Nemenman, Katia Basso, Chris Wiggins, Gustavo Stolovitzky, Ric- cardo Dalla Favera, and Andrea Califano. ARACNE: algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinformatics, 7:1-15, 2006.
Markus Maucher, Barbara Kracher, Michael Kühl, and Hans A. Kestler. Inferring Boolean network structure via correlation. Bioinformatics, 27(11):1529-155 04 2011.
Patrick E Meyer, Frederic Lafitte, and Gianluca Bontempi. minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information. BMC bioinformatics, 9:1-10, 2008.
Tristan Millington and Mahesan Niranjan. Quantifying influence in financial markets via partial correlation network inference. In Proceedings of the 11th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 306-311. IEEE CS. 2019.
Frank Noé and Feliks Nuske. variational approach to modeling slow processes in stochastic dynamical systems. Multiscale Modeling & Simulation, 11(2):35-6555 2013.
11

---

Published as a conference paper at ICLR 2024
Nan Papili Gao, SM Minhaz Ud-Dean, Olivier Gandrillon, and Rudiyanto Gunawan. SINCERI- TIES: inferring gene regulatory networks from stte-stamped single cell transcriptional expression profiles. Bioinformatics, 34(2):258-22) 2268-66,..20 2018.
Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning Python. Journal of Machine Learning Research, 12:2825- 2830, 2011.
Aditya Pratapa, Amogh P. Jalihal, Jeffrey N. Law, Aditya Bharadwaj,, and T.M. Murali, Benchmark- ing algorithms for gene regulatory network inference from single-cell transcriptomic data. Nature Methods, ((2)):4771544, 2020.
Xiaojie Qiu, Arman Rahimzamani, Li Wang, Bingcheng Ren, Qi Mao, Timothy Durham, José L McFaline- MicFalin-Figuerra, Lauren Saunders, Cole Trapnell, and Sreeram Kannan. Inferring causal gene regulatory networks from coupled single-cell expression dynamics using Scribe. Cell Systems, 10 (3):265-274, 2020.
Arman Rahimzamani and Sreeram Kannan. Network inference using directed information: The deterministic limit. In 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 163. IEEE, 2016.
Wouter Saelens, Robrecht Cannoodt, Helena Todorov, and Yvan Saeys. A comparison of single-cell trajectory inference methods. Nature Biotechnology, 37(5):547-554, 2019.
Marco Sarich, Frank Noé, and Christof Schütte. On the approximation quality of markov state models. Multiscale Modeling & Simulation, ((4):1154-117, 2010,
David J. Sheskin. Handbook of Parametric and Nonparametric Statistical Procedures. Chapman and hall/CRC, 2003.
Ravid Shwartz-Zi Ziv and Naftali Tishby. Opening the black box of deep neural networks via informa- tion. arXiv preprint arXiv: 11XXiv:103..00 0777.0003000 2017.
Stephen Smith, Karla L Miller, Gholamreza -hiimi-Khorshidi, Matthew Webster, Christian Beckmann, Thomas E Nichols, Joseph D Ramsey, and Mark W Woolrich. Network modelling methods for FMRI. Neurvimage, 54(2):875-891, 2011.
Chao Song, Youfang Lin, Shengman Guo, and Huaiyu Wan. Spatial-temporal synchronous graph convolutional networks: A new framework for spatial-temporal network data forecasting. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 914-921, 2020.
Alicia T Specht and Jun Li. Leap: constructing gene co-expression networks for single-cell ma- sequencing data using pseudotime ordering. Bioinformatics, 3361))744733 2017.
A.K Subramanian. Pytorch-vae.
2020.
Attila Szabo and Neil Ostlund. Modern Quantum Chemistry: Introduction to Advanced Electronic Structure Theory. Courier Corporation, 2012.
Bishenghui Tao, Hong-Ning Dai, Jiajing Wu, Ivan Wang-Hei Ho, Zibin Zheng, and Chak Fong Cheang. Complex network analysis the bitcoin transaction network. IEEE Transactions on Circuits and Systems II: Express Briefs, 1003)1100--1013, 2021.
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. In Proceedings of 2015 IEEE Information Theory Workshop (ITW), pp. 1-5. IEEE, 2015.
Naftali Tishby, F.C. Pereira, and W. Biale. The information bottleneck method. In Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp- 368-377. IEEE, 1999.
Masashi Tsubaki, Kentaro Tomii, and Jun Sese. Compound interaction prediction with end- to-end learning of neural networks for graphs and sequences. Bioinformatics, 35(2))309-311 2019.
12

---

Published as a conference paper at ICLR 2024
S. Varrette, H. Cartiaux, S. Peter, E. Kieffer, T. Valette, and A. Olloh. Management of an Academic HPC & Research Computing Facility: The ULHPC Experience 2.0. In Proceedings of the 2022 6th High Performance Computing and Cluster Technologies Conference. (HPCCT 2022), Fuzhou, China, July 2022. Association for Computing Machinery (ACM). ISBN 978-1-4503-9666-6.
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Courna- peau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nel- son, Eric Jones, Rohert Kern, Eric Larson, Carey, ilhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, lan Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mul- bregt, and SciPy 1.0 Contributors, SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-221 2722, 2020.
Aoran Wang and Jun Pang. Iterative structural inference directed graphs. In Advances in Neural Information Processing Systems 35 (NeurIPS), 2022.
Aoran Wang and Jun Pang. Active learning based structural inference. In Proceedings of the 40th International Conference Machine Learning (ICML), pp.
PMLR, 2023.
Aoran Wang, Tsz Pan Tong, and Jun Pang. Effective and efficient structural inference with reservoir computing. In Proceedings of the 40th International Conference on Machine Learning (ICML), pp. 3639363333000 1-36410. PMLR, 2023a.
Kun Wang, Guohao Li, Shilong Wang, Guibin Zhang, Kai Wang, Yang You, Xiaojiang Peng, Yux- uan Liang, and Yang Wang. The snowflake hypothesis: Training deep GNN with one node one receptive field. arXiv preprint arXiv:2308.1000 2023b.
Charles Ernest Weatherburn. A First Course Mathematical Statisties, volume 158. Cambridge University Press Archive, 1949.
Ezra Webb, Ben Day, Helena Andres-Terre, and Pietro Lió. Factorised neural relational inference for multi-interaction systems. arXiv preprints arXiv:1905.08721, 2019.
Joe Whittaker. Graphical Models in Applied Multivariate Statistics. Wiley, 1990.
Jianwei Yang, Jiasen Lu, Stefan Lee, Dhruv Batra, and Devi Parikh. Graph r-cnn for scene graph generation. In Proceedings of the European conference on computer vision (ECCV), pp. 670-685, 2018.
Yiming Zuo, Guoqiang Yu, Mahlet G. Tadesse, and Habtom W. Ressom. Biological network infer- ence using low order partial correlation. Methods, 69(3):266-22) 273, 2014.
13

---

Published as a conference paper at ICLR 2024
A MORE DETAILS ABOUT SIDEC
A.I PARTIAL CORRELATIONS
In this section, we provide a derivation of the partial correlation mentioned in Section 3 The derivation strictly follows the description provided by Kim (015). Consider the random vector X (I1, T2..... -..... FT, where | n. We denote the variance of a variable random æ; and the covariance between two random variables z; and vy as V Vari(= var(æ;) and Covy (= cov(x., Lz); respectively, The variance-covariance matrices of random vectors X and Xs (S C 21, n) random sub-vector of the random vector X. The correlation between two random variables 2; and 2j is denoted by Cony (= corr(x;; æz))). Thus, the partial correlation rijjk of a with æj given Iy is:
and |S| n) are denoted by Cx and Cs, respectively, where Xg
Tij VVariyVor,
Tij FakTjk
Tajlk
(14)
VI i
jk
Interestingly, we can easily obtain another version of the partial correlation as: Pajkk corr(resid resid(j(k))
Cous Coci V War Coouj
(15)
Var: Coci ar Var, CougxVart
where resid(ilk) = x2 2, (x(2)) and (xxg) Cori V ari . This form concurrence the defini- tion of partial correlation in (Whittaker 1990).
introduce the calculation of higher-order partial correlations proposed by Whittaker (1990). To do this, we need to consider the inverse coriaice-ovvariance matrix X, Dx Cx We denote d; and Cij are the (i, cell of the matrices Dx and Cx, respectively. Then following (Whittaker) 1990) the partial correlation of x; with Iy given a random vector Xg is:
With this equivalence,
dej
Tiyls
(16)
where Xs the random sub-vector of X after removing the random variables x; and x, and its size is |S| X 2. Through inversion of Dx all partial correlations between and æj given all other variables Xs are simultaneously calculated. This formulation is practice more efficient calculation than the recursion formula proposed by Anderson (958). We suggest interested readers to refer to the literature for more details. Thus the introduction of Eqn. 4 in Section is validated.
A.2 AUTOCORRELATION LOSS
As elaborated in Section 4.2 the autocorrelation loss term Lantees outlined in Equation I1 plays pivotal role enhancing the network's 's capacity to capture the extended timescale kinetics within time-series data (Hernández et al.) 2018). While the minimization of the reconstruction loss holds the potential to recover these intricate dynamical processes, it alone proves insufficient. To ensure robust model convergence, the original VDE paper (Hernández al. 2018 drew inspiration from the variational approach to conformational dynamics (VAC) (Noé & Nuske 2013). In this context, the authors introduced the autocorrelation loss function. This innovation allowed for the estimation of autocorrelations using multiple short simulations distributed across the state space, eliminating necessity for direct simulation of slow processes within a single, lengthy trajectory. This feature confers critical advantage in addressing the sampling challenges that frequently plague dynami- cal systems investigations. In line with this approach, we incorporate the autocorrelation loss into SIDEC. For more comprehensive understanding of the derivation of Equation [I] we refer to the insights offered by Noë & Nuske (2013). Following the insights, here we provide more details about the derivation of Eqn.
Basics. Let Ω be state space, and use y denote points in this state space. We consider Markov process mp; e which is stationary and ergodic with respect to its unique stationary (in- variant) distribution (xx) p(mp, = x) Vt. The dynamics of the process mp, characterized
14

---

Published as a conference paper at ICLR 2024
by the transition density
y(x,y:T)= p(mpr47 y/mp; = =x),
(17)
which we assume to be independent of the time t. The correlation density, i.e., the probability density of finding the process points x and y at time spacing of T, is then defined by
y:T, = (xx) y:r) = p(mp:++ = =, mp; x).
(18)
We further assume mp, be reversible with respect its stationary distribution, i.e.:
p(x)p(xxy;;)) = ,u(y)p(x,y;7)
(19)
y(x,y;7) C(y,y;+)
(20)
Reversibility is not strictly necessary but tremendously simplifies the forthcoming expressions and their interpretation (Sarich et al., 20T0). If at time ( O, the process is distributed according probability distribution Pu. the corresponding distribution at time T is given by:
pr(y) - dxpo(x))(xxx)))
P())
(21)
The time evolution of probability densities can be seen as the action of linear operator P(T), called the propagator of the process. This well-defined operator on the Hilbert space 22 (92)
of functions which are square-integrabe with respect to the weight function μ² The scalar-product on this space is given by
(ulv), /
(22)
we assume the transition density to be a smooth and bounded function of x and y, the propagator can be shown to be bounded, with operator norm less or equal to one. Since the stationary of μ implies P(T)μ = μr, we have [[P(T)] = 1. Reversibility allows us to show that the propagator is self-adjoint and compact. Furthermore, using the definition of the transition density, we can show that (r) satisfies Chapman Kolinoorroo equation: For times T1, T2 2 O, we have:
P(n + 72) P(T))(
(23)
Spectral decomposition. It follows from the above arguments that P(T) possesses sequence of real eigenvalues (T(7) with [T) S 1 and [>x(T) - O. Each of these eigenvalues corresponds to an eigenfunction 4 L (2). The functions 4 form orthonormal basis of the Hilbert space " (52). Clearly, (1 = is an eigenvalue with eigenfunction = μ. In many applications, we can assume that (T is non-degenerate and -1 is not an eigenvalue. Additionally, there usually is number of positive eigenvalues
1 (T > >2(T) >..>>>..>>> (m)
(24)
which are separated from the remaining spectrum. Because of the Chapman- equation, each eigenvalue (T( decays exponentially in time, i.c. we have
>x(T) exp(-Ke7),
(25)
for some rate K; 2 0. Clearly, Ki = 0, .., Kim are close to zero, and all remaining rates are significantly larger than zero. If now expand function (12) in terms of the functions 4, i.c.
(26)
we can decompose the action of the operator P(T) into its action on each of the basis functions:
P(r)u
(x(r)(ul))))
(27)
Σ exp
=
15

---

Published as a conference paper at ICLR 2024
all except the first terms in the above have become very small (Sarich al. 2010), and to a good approximation we have
For time windows km+1''
P(T)us R
E (ul)))
(28)
Knowledge of the dominant eigenfunctions and eigenvalues is therefore most helpful to the under- standing of the process.
Rayleigh variational principle. In nontrivial dynamical systems neither the correlation densities D(x.y:7) and Y: nor the eigenvalues (x(T) and eigenfunctions 1 are analytically avail- able. Noé Nuske (2013) provides a variational principle based on which these quantities can be estimated from simulation data generated by the dynamical process mp,, For this, the for- malism introduced above used to formulate the Rayleigh variational principle used in quantum mechanics (Szabo & Ostlund| 2012) for Markov processes. Let f be a real-valued function of state, R. Its autocorrelation with respect to the stochastic process mp, is given by
f = f(() Ω
Jel, dxdy (x))x
autoc(f: = (f((m)))
(P(r)ufluf), 1. (29)
In the Dirac notation often used in physical literature, integrals such the one above may be abbre- We may easily get that the autocorrelation function of
viated by [xx((x))((xx) = (n/(P(())luf)
weighted eigenfunction Ta - 2 is:
autoc((::T) = B[rx (mp)))
= (P(T)lalke)a>
(30)
=(u())(e(le)pp))
= -d((T),
where du (T) is the eigenvalue of rk-
Let l2 be an approximate model for the second cigenfunction, which is normalized and orthogonal
to the true first eigenfunction: (12, 14) 1) 0 and (2) = 1. Then according to (2.24) in Noo & Nuske 2013). we find for 12 = p 12: (1p))
autoc(F2;
S (2(r).
(31)
Similarly, let l be approximate model for the k'th eigenfunction, with the normalization and orthogonality constraints: 4)) 0, vi < k and (ie, 44), then
autoc(fx;:) = [F (mpo)fi(mpp)) < (T).
(32)
An important insight at this point is that variational principle of conformation dynamics can be formulated in terms of correlation functions. In contrast to quantum mechanics other fields where the variational principle has been successfully employed, no closed-form expression of the operator P(T) is needed. The ability to express the variational principle in terms of correlation functions with respect to Prr) means that the eigenvalues to be maximized can be directly estimated from data. If statistically sufficient realizations of mp, are available, then the autocorrelation function can be estimated via
autoc(fx:1) r) ((fx(mp))))) (mp,))
Σ Fx (mp)))
(33)
where N is the number of simulated time windows of length
For Structural Inference. This section provides the contextual foundation for the formulations presented above. The variational principle posits that, in the limit of an infinite dataset, no process within the data can identified that is slower than the true underlying process. However, our aim is to encode the dynamics of individual nodes into latent variables, capturing the true essence of the dynamical system's 's behavior. Therefore, we must establish a metric to gauge the suitability of the current VDE in modeling the true process. This metric ensures that the latent variable z, effectively encapsulates the dynamics that faithfully represent the entire process.
As expressed in Equation 28 acquiring knowledge of the dominant eigenfunctions I, and their cor- responding eigenvalues X, proves most valuable for approximating the underlying process. Conse- quently, turn the Rayleigh variational principle estimate these critical quantities from data
16

---

Published as a conference paper at ICLR 2024
Algorithm From temporal order to directed edges.
1: Input: M sets of latent variables as Z = [m]] S m Σ M) where [mj E (T +)
2: Output: A set of matrices of p-values representing the relation between n nodes: (P),
3: for node in total nodes do
4: Zimm], = In 2
zi-1 .... / 2 (-1
5: end for
6: Constitute set Z = [m]] S m < M) with each [mm] = [Z(m) E RR (T(T-T-1)) 1)xm 1 sis
n).
7: for trajectory in total M trajectories do
8: for node in total n nodes do
9:
Calculate the variance-covariance matrix Cx based on [m)
Calculate Dx = Cx'
11:
for node j do Calculate partial correlation rijls
12:
de Va.
d3i
(Eqn. 4
13:
Calculate 4ajss St
11-2- 1-riy1s"
[tey sl. H - n).
14:
Calculate Pijis 24, end for
15:
16:
Form P [mm]s based on PiylS+
17:
end for
Form P(m]] ((mm)(., = (t)) for all node
19:
end for
20: Return the set of p-value matrices (P) (P [m], (Pmm) S m $ M).
generated by the dynamical process, simultaneously striving to maximize the eigenvalues. As out- lined in Equation 33 the maximization of eigenvalues aligns with the maximization of the estimated autocorrelation function. Notably, in our specific case characterized by a single linear decomposition of the phase-space limit, only eigenvalue warrants consideration. This eigenvalue corresponds as elucidated in Section 4.2
to the autocorrelation of the latent variable Zt, giving rise to Equation 11
A.3 FROM TEMPORAL ORDER TO DIRECTED EDGES
In this section, we discuss more about the method mentioned in Section 4.3 help PCOR to reconstruct the structure of directed graphs. PCOR is initially designed to measure the correlations between variables. Thus cannot provide the directionality of the relation between variables, as the calculations of correlation has no directionality. However, if we take the temporal information into consideration, and would figure out that there actually exists a temporal directed connection in the time series. Suppose we have three nodes, i, j. and k. In the underlying interaction graph, there exists a directed edge from j to i, while the node i is disconnected from the rest. We sample the node features within time period T, and obtain (V) (V", V VT-1} as the trajectory of the system. At time step 4, we have V = (uf, uj, )
which leveraged to
If just node features the same time step for PCOR calculation, it would be impossible to figure out the directed edge from j to i, as the partial correlation coefficients Pyjs PA However, we may recompose the time series which serves as the input for PCOR. Instead of feeding PCOR with {V} = {v" V1 .... VT-1J, whose element V = {or, uf. } we rearrange the node features for every time step. For example, if we would like figure out which other node may VT-I), have a directed edge to node i, we construct new feature trajectory [V}} = [V, VT?....VT-I1
_ (-11) where at time step V = {uf. 1/ - ). In this formulation, every present feature of node i is paired with the feature from the previous time step of other nodes. We then feed (Vi) to PCOR to obtain P, representing the measurement of the relation between node and other nodes. If we detect there is a connection between j and i in the obtained P,, because the events typically occur in chronological order and with the pairing of node features, the result can only lead to there is a directed edge from to í. For other nodes, we calculate the P,, Ps following the mbotenmentioned steps. After obtaining all coefficient matrices, we constitute the final coefficient matrix for the entire system P as P(. i.) We may replace of in the above formulation with zf, and thus
17

---

Published as a conference paper at ICLR 2024
Algorithm 2 Overall pipeline of SIDEC.
1: Input: M trajectories of node features V (V [mm],1 m M). Each trajectory consists of the features of nodes spanning from time [0, T 1]. which is marked 'fmj, for node i time é in trajectory m.
2: Parameters: Number of epochs E, batch size B, time window length T,
3: Output: A set of matrices of p-values representing the relation between nodes: (P),
4: Flatten all node features from all node trajectories and form a new set Van-
5: Constitute N2 batches from Vall with size B,
6: for batch X, in total N₂ batches do
7: Train encoder ENC and obtain {uf. for current batch (Eqn. 6).
8: Obtain {z} based on reparameterization step (Eqn. 7
9: Train decoder DEC and obtain {ū]t)) for current batch (Eqn.
8).
10:
Calculate reconstruction loss: Lrecom
Σ | + 1|?
Σ (1 + log 0² - 0²_ -1²)
2T
B(( -D'
11:
Calculate autocorrelation loss: Lautoe
(Eqn. 11,
12: Calculate loss for VDE: LYDE = Lrecon + Lautoe.
Update weights of encoder and decoder,
14: end for
After training, feed Van into VDE again, and collect the ž == (n} for every input,
16: Reshape 7 according to the length of the trajectory and the number of nodes, and obtain Z
[Z(m] 1 S m < M) where Z[m] E [
17: Follow steps in Algorithm to obtain the set of p-value matrices (P} = (P mm) 1 m S M).
18: Return the set of v-auuue matrices {P} [Pmm), & m S M}.
obtain the procedure mentioned in Section 4.3
We summarize the pipeline for inferring directed
edges with PCOR our work in Algorithm
A.4 THE LENGTH OF TIME WINDOW
In the context of the paper, when it is mentioned that represents the length of the time window required for the system's dynamies to be considered Markovian, it is referring to the concept of Markovianity in continuous-time Markov processes (Noé & Nuske, 2013). In these processes, the future state of the system is determined solely by its current state and not by how arrived that state. This property is essential to simplify the modeling and analysis of dynamical systems.
The time window T is crucial because it sets the scale at which the dynamics are examined. For a system be considered Markovian, the choice of 7 should be such that the memory of past states does not significantly influence the future states beyond this time window. In practical terms, this means that if you know the state of the system time 7, then this information is sufficient predict its state at time + 7 without needing to know its earlier history.
The impact of the time window the Markovian nature of the system can be understood
T is too short, the system's evolution may still influenced by its immediate past, not adhering to the Markovian property.
2. If is appropriately chosen, it allows for the dynamics of the system to 'forget' its past states, thus adhering more closely to the Markovian assumption. This is critical for the effective appli- cation of Markov models and for simplifying the analysis of the system's dynamics. This could be the case of analyzing open quantum system which somehow expresses its non-Markovian characteristic.
In these scenarios, a F value larger than might be more appropriate. the case of the StructIn- fer (Anonymous) 2023) datasets, which generated with smaller time intervals and subsequently subsampled, a value of suggests that the system behaves Markovian over these intervals. This means that past states do not significantly influence future states beyond this time window.
18

---

Published as a conference paper at ICLR 2024
Exploring how the length of the time window affects various dynamical systems and sampling fre- quencies in detail will deepen our understanding of the VDE and consequently broaden the applica- tion scenario of SIDEC. We plan to delve into this research area in future work.
A.5 FULL TRAJECTORIES OR JUST ONE?
worth investigating the effectiveness of SIDEC with just one trajectory as input and with all trajectories input. We gathered experimental data compare the performance of SIDEC with full node trajectories (SIDEC(+Full) and single-node trajectories. Table presents our findings.
Table 3: Average AUROC Results (%) of SIDEC and SIDEC(*Full) VN_SP trajectories.
Methods
VN.SP.15
VN.SP.30
VN.SP.50
VN.SP.100
VN.SP.150
VN.SP.200
VN.SP.25M
(FuEC("Full)
9.m
96.0=
95.0=
94.62
OOM
OOM
SIDEC
97.6-
97.0=
96.5+
95.7=
95.3+
95.5=
95.0+
The results indicate that SIulC((Full) is consistently outperformed by SIDEC using single-node trajectories. Additionally, using full trajectories leads to scalability issues on graphs with more than 150 nodes. It also complicates the identification of latent variables corresponding to specific nodes. These findings justified our choice to focus on single-node trajectories in the VDE stage for efficiency and efficacy.
A.6 OVERALL PIPELINE
We summarize the overall pipeline of SIDEC in Algorithm
Algorithm
Pseudocode for the encoder in
Algorithm 4 Pseudocode for the decoder in VDE.
VDE.
Input: features imput
Input: features input.
2: minp())
2: x mip5(input)
3: X = mlp2(x)
X mlp6(x)
4: mlp3(x)
X mlp7(x)
5: out mlp4(x)
out = mlp8(x)
6: Return: out
Return: out
A.7 IMPLEMENTATION
In this section, provide details about the implementation of the proposed SIDEC. Besides the description in this section, readers may also refer to the code in the following GitHub Repository: https: // cchub.com/mwe httbbb/////waag22003333 wtt.222000O/TTSCtrrO0 torch
Implementation of VDE. The basic class of VDE is implemented the inheritance of VAE Class of "PyTorch-VAE"" Subramanian 2020), with the following modification:
1. New encoder and decoder: We implement the encoder and decoder according to the VDE implementation [Hernández et al. 2018). We enclose the setup of the encoder and decoder as pseudo code and shown in Algorithms and The implementation of MLP in the encoder and decoder described in Algorithm 5 with parameters shown in Table 4
2. Changes in loading data: We integrated the data loading pipelines from iSIDG implemented by StructInfer (Anonymous] 2023) into the VDE data loading pipeline.
New loss functions: Besides the reconstruction loss term that is implemented already in the VAE Class, we implemented the pipeline for autocorrelation loss and the summation of both reconstruction loss and autocorrelation loss.
Addition of "predict()" method: We added method in the VAE class to run the trained model again and sample the latent variables.
19

---

Published as a conference paper at ICLR 2024
Algorithm 5 An MLP block.
1: Input: features inpul
2: Linear(input)
3: = Swish(x)
4: out DropOut(x)
5: Return: out
Table 4: Parameters in the Linear layers in MLPs in encoder and decoder.
BLOCK
Number of Units
Dropout Rates
MIP1
2048
MLP2
2048
0.3
MLP3
2018
0.3
MLP4
1
0,0
MLP5
2048
0.0
MLP6
2048
0.3
MI.77
2048
0.3
MLP8
Original Dimension d
0.0
Calculation of Partial Correlation Coefficients. The main pipeline of the calculation of partial correlation coefficients follows the one provided httts::/gisst sut.ggithub com fabi anp/ 939620441 9c b7 8d33ff. which is Python implementation of the calculation of partial cor- relations. But we need to calculate the p-values, thus we extend this function with the following modification.
Add "args": We added new variable input named "args", which serves as the input variable of the general control arguments.
Create a new function of ..ttst_ffissh( This function follows the one used in Scipy (Virtanen eet al. 2020), which takes in the correlation matrix and degree of freedom to calculate the t- statistics and p-values.
3. AUROC calculation: We created function caleulate the AUROC results of the constructed structure. The implementation done with the help of "roc_auc pooe_uc_sccores score()" from sklearn (Pedregosa al. 2011).
Experimental Setting. We run SIDEC single NVIDIA Tesla V100 SXM2 graphic card, which has 32 GB of graphic memory and 5120 NVIDIA CUDA Cores. The batch size is set as 256, but believe it can be larger to even 2048. The learning rate is set 0.0005. The maximum epochs for training 500.
Minor Impact on Running Time. Our method, SIDEC, integrates structural inference through two main components: training of VDE and subsequent inference utilizing both VDE and PCOR. Through our experimental observations, we found that the training phase of VDE is the most time- consuming aspect, accounting for approximately 98% of the total computation time. This pattern is consistent across all our experiments.
B MORE DETAILS ABOUT DATASETS
StructInfer benchmark. The StructInfer benchmark (Anonymous 2023) evaluated 12 structural inference methods comprehensive way on synthetic dataset. The dataset firmly believed as created by the author, and it covers 11 types of different underlying interaction graphs and two types of dynamical simulations. We name the dataset StructInfer, which is the same as the name provided on the official website httee:tttt/// htttps:/sstruttinnnst http:/stuutnfffet git.ub...tbb.b......... io/ As there are so many trajectories, we chose the ones under the name "Vascular Networks" or in short "VN",, whose un- derlying interaction graphs approximate the wollwwoldd vascular networks in bio-systems. Besides, we also contact the authors and get the trajectories with more than 100 nodes, and the ones with additive Gaussian noise. We would like to thank the authors for their help. As the data is already
20

---

Published as a conference paper at ICLR 2024
split into three sets: for training, for validation, and for testing, we keep this setting. In the follow- ing paragraphs, we describe more details about the Springs and NetSims simulations utilized by the StructInfer benchmark.
For Springs simulation, it follows the approach by (iipl et al. 2018). to simulate spring connne-onneectd particles' motion in 2D box using the Springs simulation. In this setup, nodes represent particles, and edges correspond to springs governed by Hooke's 's law. The Springs simulation's 's dynamics are described by a second-order ordinary differential equation: my (1) ==eNx; -k- -k=(2 (1) Here, m, represents particle mass (assumed as 1). is the fixed spring constant (set 1), and N, is the set of neighhoring nodes with directed connections to node i, which is sub-sampled from the graphs generated in the StructInfer in previous steps. We integrate this equation compute xx( and subsequently (t) for each time step t. The resulting values of (t) and x; (t) create 4D node features each time step.
For NetSims simulation, it is firstly mentioned in NetSim dataset (Smith et al., 2011), which offers simulations of blood-oxyygen-eeee seygen-level-deeender (BOLD) imaging data various human brain regions. Nodes in the dataset represent spatial regions of interest from brain atlases or functional tasks. In- teraction graphs from the previous section determine connections between these regions. Dynamics x,(() + C M4, where controls temporal smoothing and neural lag (set to 0.1 based on (Smith et al. (201T) and C' regulates external input interactions (set to zero to minimize external input noise) (Smith et al. 2011). ID node features at each time step are obtained from the sampled r1(().
are governed by a first-order ODE model: (t) = a Σjex, (y(()
For trajectories with Gaussian noise, we refer to the ones from StructInfer dataset (Anonymous) 2023), which we utilized in our study, includes trajectories with Gaussian noise added at various levels. As detailed in the dataset's documentation, the authors introduced this noise to the generated trajectories to simulate real-world conditions more accurately. The modified node features, repre- sented as of, are calculated using the formula: of = of + 0.02 A, where Ç is standard Gaussian random variable (Ç- ~ (0, 1)), of denotes the original feature vector of node i at time and A rep- resents the designated noise level. The noise levels applied range from 1 to 5 across all the original trajectories.
Synthetic networks. The six directed Boolean networks (LI, LL, CY, BF, TF, BF-CV) are the most often observed fragments in many gene regulatory networks, each has 7, 18, 6. 7, 8 and can acknowl-
10 nodes, respectively. Thus by carrying out experiments on these networks, edge the performance of the chosen methods on the structural inference of real-world biological networks. We collect the six ground-truth directed Boolean networks from Pratapa et al. (2020) and simulate the single-cell evolving trajectories with BoolODE Pratapa et al. (2020) (https: 7/github. www.ccoo/uuuaaa. waaerraai-rroup/pee Roopp//ooolloooo ODE) with default settings mentioned in that paper for every network. We first sample total number of 12000 raw trajectories. We then sample different numbers of trajectories from raw trajectories and randomly group them into three datasets: for train- ing, for validation, and for testing, with ratio of 2 2. After that, sample different numbers of snapshots according to the requirements experiments Section 5.1 with equal time intervals in every trajectory and save them as ".npy" files for data loading.
C IMPLEMENTATION OF BASELINES
C.I NRI
We use the official implementation code by the author from https://github. comb ethanfetaya /aNRRa with a customized data loader for our chosen datasets. We add our metric evaluation in the "test" function, after the calculation of accuracy in the original code.
C.2 MPM
We use the official implementation code by the author from https://githuit com/ hilbert9221/NRI- with customized data loader for chosen datasets. We add our metric evaluation for AUROC in the "evaluate()" function class "XNRIDECIns" in the original code.
21

---

Published as a conference paper at ICLR 2024
C.3 ACD
We follow the official implementation code by the author as the framework for ACD (ttpps: Amorti zedCausa liiaaDiissssss We the code with a cus- tomized data loader for the datasets in this work. We implement the metric-caleulation pipeline in the "forward pass and eval(" function.
7/github.
C.4 ISIDG
We ask the authors of iSIDG for the code and follow the instructions the settings of hyper- parameters their work. We disable the metric evaluations for the AUPRC and Jaccard index the original implementation of iSIDG for faster computation.
C.5 RCSI
We would like to thank the authors of RCSI for the code. We follow the instructions the settings of hyper-parameters in their work. Same as iSIDG, we disable the metric evaluations for AUPRC and Jaccard index in the original implementation of iSIDG for faster computation.
C.6 TIGRESS
In utilization of TIGRESS, adhere to the official implementation provided by the author, cessible at utltttss://githi com/ ipvert/tigress https////e..nnn.oooooo//tttrreee
To streamline our experimental work- flow, have developed specialized wrapper. This wrapper exhibits the capability to parse multi- ple arguments, thereby facilitating the selection of specific trajectories for focused inference. Addi- tionally, it seamlessly transforms trajectories into format compatible with TIGRESS, orchestrates the systematic input of trajectories into the TIGRESS algorithm, and efficiently archives the resultant outputs in predefined directories.
C.7 ARACNE
We the implementation of ARACNe by the Bioconductor (Huber et al. 2015) package minet (Meyer et al. 2008) with a customized wrapper for the selection of a set of targeted tra- jectories for inference, transform trajectories into suitable format, feed each trajectory into the ARACNe algorithm, and store the output into designated directories.
C.8
CLR
For our implementation of CLR, we harness the capabilities the minet package, a Bioconductor resource (Huber al. 2013 Meyer et al. 2008). To optimize workflow, have developed tailored wrapper. This wrapper has the ability to parse multiple arguments, enabling the selection of specific trajectories for targeted inference. Furthermore, it facilitates the seamless transformation of trajectories into a compatible format, orchestrates the sequential input of trajectories into the CLR algorithm, and efficiently stores the resulting outputs designated directories.
C.9 PIDC
We the official implementation of PIDC by the author https: / com/ Rhhaaneers/Newwokknnteio http///wwwwweeeeeeeeeeee
with customized wrapper for data loading. Our wrap- per will parse multiple arguments select set of targeted trajectories for inference, transform trajectories into a suitable format, feed each trajectory into the PIDC algorithm, and store the output into designated directories.
C.10 SCRIBE
Our utilization of Scribe relies the official implementation provided by the developer, which is accessible at wetttts:/// //qit/bb///bb.././//// com/ htaa///riittoeell///ieeeeee To streamline our work- flow, have developed custom wrapper. This wrapper is designed perform range of tasks,
22

---

Published as a conference paper at ICLR 2024
including the selection of specific target trajectories for inference, the transformation of trajecto- ries into a compatible format, the sequential input of trajectories into the Scribe algorithm, and the systematic organization and storage of output results within designated directories.
C.I1 DYNGENIE3
Our usage of dynGENIE3 relies on the official implementation authored by the developer, acces- sible at h /tttps:/ggitub.. com/ waauynb/dynnEN/e
To streamline the process, have designed a custom wrapper. This wrapper performs various tasks, such as the selection of tar- get trajectories for inference, data format conversion ensure compatibility, sequential input of trajectories into the XGBGRN algorithm, and systematic storage of output results in designated directories.
C.12 XGBGRN
We employ the official XGBGRN implementation developed by the author, accessible at https:: 7/github. com/ lab31 CRRNs_noni inear__DEs, within a customized wrapper. This wrapper serves multiple purposes, including the selection of specific trajectories for inference, the transformation of these trajectories into compatible format, the sequential feeding of each trajec- tory into the XGBGRN algorithm, and the organized storage of output results in dedicated directo- ries.
D MORE EXPERIMENTAL RESULTS
Because of the page limitation, more experimental results can be found in this section.
D.1 PERFORMANCE ON RANDOM GRAPHS
One of the key advantages of SIDEC lies in its absence of a uniform prior edges, distinguishing from previous VAE-based structural inference methods (Kipf et al., [0088] [Cen al.) [022]] Löwe et al. 2022) Wang & Pang 2022, Wang al., (203a)), which incorporate uniform prior in their Ki-divereence terms. This distinction prompted our interest in assessing SIDEC and the baseline methods on trajectories where edges possess fixed probability of existence absence. For this purpose, we selected the "Social_Netwee Networrs" trajectories from the StructInfer benchmark, character- ized by underlying graphs generated via Erdös-Rényí graph (ERDdS R&wi 1959) generators, featuring directed edges generated with a fixed probability.
Our experimentation involved running SIDEC and the baseline methods "SN_SP" and "SN_NS" trajectories, each with 15, 30, 50, and 100 nodes. The results, presented as the average AUROC values from runs, are detailed in Table noteworthy observation from the table is that when tasked with structural inference on trajectories produced by Springs simulations, featuring four- dimensional features, the performance differences between SIDEC and VAE-based methods are less pronounced compared to the results outlined in Section 5.2
This phenomenon is particularly evident when the graph is relatively small, containing no more than 30 nodes. This aligns with the notion that VAE-based structural inference methods, equipped with the assumed uniform prior and the capacity to extract information from multi-dimensional features, exhibit strong performance in scenarios where the underlying interaction graph resembles Erdös- Rényi graph and the trajectories exhibit multi-dimensional characteristics.
However, as the graph size increases, SIDEC consistently outperforms VAE-based methods, ben- efiting from its scale-insensitive nature. Furthermore, when trajectories solely comprise one- dimensional features, SIDEC surpasses VAE-based methods significantly. In such cases, the in- corporation of VDE for dynamic encoding. in conjunction with the synergy between PCOR and VDE, equips SIDEC to effectively address information deficiencies within the trajectories.
23

---

Published as a conference paper at ICLR 2024
Table 5: Average AUROC Results (%) of SIDEC and baselines on SN trajectories.
Methods
SN-SP_15
SN.SP-30
SN_SP_50
SN.SP_1ON
NRI
93.3= 0:01
79.9 0.02
80.4 0.22
71.8 0.01
MPM
92.7= 0.00
79.3+ 0.01
75.9 0:1
69.4
ACD
93.5= 0.01
81.2 0.01
79.6 0.02
68.8 0.02
iSIDG
93.5 0.00
81.42 0.01
80.8 0.02
69.3 0.01
RCSI
94.1 0,02
82.7 0.01
81.3 0.01
73.44: 0.02
SIDEC
94.0=
89.8
89.0
90.0
Methods
SN NS 15
SN NS 30
SN NS 50
SN NS 100
NRI
58.4= 0.04
51 = 0.01
49. 19.6 0.83
50.2 0.83
MPM
67.4 0.02
50.9 0.01
53.1 + 0.03
50.1 + 0,02
ACD
65.2
52.9
49.3 0:022
50,8
iSIDG
66. + 0.94
53.8+ 0.05
0.01
51.7 0.02
RCSI
67.6= 0,03
55.8 0.02
55.0 = 0.022
53.2 0.03
SIDEC
91.6 0,02
90.2 0.02
89.7 0.02
89.2 0.02
D.2 PERFORMANCE ON UNDIRECTED GRAPHS
As established at the outset of this paper, numerous correlation- structural inference methods are ill-suited for the reconstruction of directed graphs. While our prior experiments exclusively employed trajectories with directed underlying interaction graphs, it's equally pertinent to assess techniques, traditional baselines, and our proposed SIDEC on
the performance of correlation-bes trajectories generated with undirected underlying interaction graphs. Fortunately, the StructInfer benchmark contains such trajectories, specifically under the category [Landscape.Networks: fea- turing undirected underlying interaction graphs. In addition to the aforementioned baselines, we have incorporated several other methods for comparative evaluation:
TIGRESS (Haury et al. 2012): : correlation- based model employing iterative feature selection
with least-angle regression and bootstrapping.
method that calculates pairwise mutual information and utilizes the Data Processing Inequality principle to eliminate indirect in- teractions.
ARACNe (Margolin et al., 2006); A
CLR (Faith et al., 2007): This method employs pairwise mutual information but assumes back- ground noise distribution for mutual information.
PIDC (Chan et al., 2017): A model based on Partial Information Decomposition, interpreting aggregated unique information as the strength of edges between nodes.
Scribe (Qiu et al.] 2020): Utilizes Restricted Directed Information (Rahimzamani & Kannan
2016) and its variants to quantify causality within the structure by considering the influence of confounding factors.
dynGENIE3 HHuynh-Thu & Geurts 2018): A random-forest-bee based method that employs ordinary
differential equations to model time series dynamics.
XGBGRN (Ma et al. 20200: Similar to dynGENIE3 but leverages XGBoost Chen & Guestrin 2016) instead of random forests.
It's essential note that these methods operate exclusively on one-dimensional features and cannot perform structural inference on trajectories generated using Springs simulation. Additionally, their design inherently prevents them from inferring directed edges. In our evaluation, we' also imple- mented an algorithm based solely on partial correlation coefficients, denoted as PCOR, and a variant of SIDEC that lacks the component for rearranging latent variables to reconstruct directed graphs, structural infer-
known as SIDEC-V. Besides that, we also implemented following correlation ence methods: Person Correlation (P-cor) (Maucher et al.) 2011), Distance Correlation (D-cor) (Liu) al. (2021), Low-order Partial Correlation (LPCOR) (Zuo et al. 2014), and Semi-pane partial Correla- tion Coefficients (SPCOR) (Kim 2015). These methods, by design, only produce symmetric results. We show the results in Table. 6 On "LN.SP" trajectories, SIDEC and SIDEC-V V consistently out- perform VAE- based methods, displaying a robustness independent of graph size, with minor drops in AUROC values. This trend aligns with findings in Section[5.2
However, when examining the results for "LN NS" trajectories, SIDEC exhibits superior perfor- mance mainly on graphs with fewer than 30 nodes. It's worth noting that the performance
24

---

Published as a conference paper at ICLR 2024
Table 6: Average AUROC Results (%) of SIDEC and baselines on LN trajectories.
Methods
LN-SP_15
LN.SP_30
LN.SP 50
LN.SP_100
NRI
97.0=
94.9 0.0
87.1# + 0.1
82.8# 001
MPM
97.9=
95.5+
86.9+
84.2+
ACD
97.0= 0.02
95.8 not
87.6# 0.02
83.9 0,2
iSIDG
97.41 0,02
94.7.1 0.02
87.4 0.02
83.2: 25 0.02
RCSI
97.32 0.02
94.41 0.02
88.0 0: 0.02
84.31 0.02
SIDEC-V
97.6
96.33
96.0a O:t 0.02
94 0.02
SIDEC
97.7.
96.5#
96.2+ nu02
95.1
Methods
LN_NS_15
LN_NS_30
LN LN_NS_50
LN_NS_100
TIGRESS
84.1# 1,16
87.4+ 3332
92.2+ 2:2 0.42
93.8 196
ARACNe
92.3+
80.4+
71.2+
62.8 +
CLR
97.34 3.17
96. 4:57
91.0# 2.35
95.0# 0.33
PIDC
97.5+ 1.01
82.0+ 7:28
88.6+ 1.69
94.2 2.28
Scribe
54.2: 3.98
56.2+ 3.88
52.1+ 2.49
52.51 1.62
dynGENIE
5123
50.1
50.5a
67.3
XGBGRN
97.2: 1.13
96.51
96.91 0.83
98.0 0.93
NRI
56.0 0.94
53.9 0.02
54.4 0.02
51.8 0.03
MPM
52.2 0,02
62.1 0.05
53.4+ 0.01
50.4 40.4+000 0.03
ACD
61.9# 0.03
61.6 0.04
53.4+ 0.02
50.2 0.02
SIDG
59.2#
56.2
55.7
52.3# 3
RCSI
60.3# 0.03
60.2 0.02
57.6.t 0.03
53.5 0.02
P-cor
68. 9 0.31
65. 65.1+ 0.10
66.7 0.39
68. 0.44
D-cor
67.32 0.23
66.4
65.22 0.20
65.0= 0.26
LPCOR
81.9 0.60
80.61 1.56
78.52 0.98
69.52 301
SPCOR
95.0=
91.8:
84.0= 0.63
71.5
PCOR
97.3# 3tt 0.56
92.0+ 5:20
84.8 1.66
79.2 4.52
SIDEC-V
97.2+ 0,05
93.8+ 0.03
96.0+ 0.01
96.0 0,02
SIDEC
97.4# 0.05
96.7 0.02
97.0 0.83
97.5# 0,02
gaps are relatively narrow, with XGBGRN, CLR, and PIDC maintaining high accuracy across the datasets. Nonetheless, these methods display substantial result deviations, indicating limited stabil- ity and reliability reconstruction. Furthermore, XGBGRN, CLR, and PIDC cannot accommodate trajectories with multi-dimensional features, as they conflict with their core operating mechanisms. The results indicate that both P-cor and D-cor show subpar performance across all datasets. This can be attributed to their limitation of considering only pairwise correlations and overlooking the influence other potentially impactful nodes. LPCOR, while better, still falls short as it calculates subset rather than all other variables.
coefficients conditional
Interestingly, SPCOR and PCOR demonstrate comparable performance, with PCOR slightly out- performing SPCOR. This be attributed to SPCOR's reliance on semi-partial correlation cal- culations, which may limit its effectiveness in identifying connections between nodes with larger degrees. Despite the variations in graph structure across the datasets, neither SPCOR PCOR outperforms SIDEC. In contrast, SIDEC boasts broader applicability, irrespective of node feature dimensionality, and consistently delivers medium to high accuracy. In comparison PCOR, which can be regarded the second half of SIDEC, our proposed method markedly enhances structural reconstruction accuracy, underscoring the value of encoding node dynamies in structural inference.
D.3 PERFORMANCE ON LARGER GRAPHS
To bolster our claim of SIDEC's s scalability, we conducted additional experiments using the NRI simulator (Kipf al.) 2018 tool also employed by (Chen et al. [021 Löwe et al. 2022 Wang & Pang 2022 Wang et al. 2023a Wang Pang 2023) for dataset generation. These experiments involved generating spring-bal ball systems with 500 and 1,000 balls, producing a total of 12,000 tra- jectories per graph, subsequently divided into training, validation, and test sets an 8:2:2 ratio.
In these experiments, only SIDEC able to operate effectively without encountering out-of- memory (OOM) issues, unlike other VAE-based baseline methods. The resulting average AUROC values ten runs shown in Table 7 Compared with other methods, SIDEC managed
25

---

Published as a conference paper at ICLR 2024
Table 7: Average AUROC Results (%) of SIDEC spring-bal. systems.
500 Balls
1,000 Balls
SIDEC
92.8+ 0:4
90.1+ 0.0
Table 8: Statisties of PEMS datasets.
Dataset
Nodes
# Edges
# Time Steps
Missing Ratio
PEMS03
358
547
26, 208
0.672%
PEMS04
307
340
16, 992
3.182%
PEMS07
883
866
28, 224
0.452%
Table 9; Average AUROC Results (%) SIDEC on PEMS datasets.
PEMS03
PEMS04
PEMS07
SIDEC
72.8+
75.4=
70.1+
construct the structure of these graphs with 500 and 1,000 nodes. And compared with the results of it smaller graphs, the results on the AUROC values still support its high accuracy the struc- tural inference. These results not only demonstrate SIDEC's capability to reconstruct the structure of large graphs but also maintain high accuracy in structural inference across varying graph sizes.
D.4 PERFORMANCE ON REAL-WORLD DATASET
We would like to emphasis that obtaining comprehensive, real-world data for structural inference poses significant challenges, both terms of cost and time. Often, real-world data lack complete information on node features reliable knowledge of underlying interaction graphs, seen in gene regulatory networks (Pratapa et al. 2020) and scene graphs (Yang et al. 2018). other cases, such as transaction networks (Tao et al. 2021) and road maps (Biagioni & Eriksson 2012). data may not cover all nodes in the system.
Despite these challenges, agree with the necessity of testing method, SIDEC, real-world data. To this end, we have conducted additional experiments using three widely recognized public traffic network datasets: PEMS03, PEMS04, and PEMS07 (Song al., 2020). These datasets, derived from the California Caltrans Performance Measurement System (PeMS) (Chen et al. 2001), comprise data aggregated into 5-minute intervals. The adjacency matrix of the nodes constructed by road network distance with a thresholded Gaussian kernel (Song et al. 2020). Table summarizes these datasets.
We resampled the data such that constructing 49 time steps of points for each trajectory. and obtained 534, 346, and 547 trajectories, respectively. We then split all datasets with ratio 8:2:2 into training sets, validation sets and test sets, and run SIDEC on all of them. Our results, presented in Table 9 demonstrate SIDEC's effective performance despite challenges such as noise, missing data, and adjacency matrix inaccuracies. It's important to note that these datasets' adjacency matrices only connect sensors on the same road, omitting alternative connecting paths, which could impact results. As shown in the table, despite the noise and missing data, and the inaccuracy in the adjacency matrices provided by the dataset, SIDEC manages to perform well and can deal with the challenges. It is notable that the adjacency matrices in these datasets just connect the sensors if they on the same road, but they ignore the existence of paths or alleys that may also connect the sensors. Because of this, when compared with the experimental results discussed in Section the performance of SIDEC is impacted.
E
LIMITATIONS
While SIDEC showcases considerable capabilities in structural inference, akin to other methods in this domain, it does have certain limitations that warrant attention.
First and foremost, SIDEC is tailored for dynamical systems characterized by a static underlying interacting structure. In scenarios where the underlying structure undergoes temporal variations,
26

---

Published as a conference paper at ICLR 2024
the simplicity of the VDE setup hinder the effective capture of node dynamics. However, envision the possibility extending SIDEC to accommodate dynamic graphs by encoding edge dynamics, thus addressing this limitation.
Secondly, the dynamical systems under investigation must be fully observed within the defined time period. This entails that all nodes need to be observed, and their features meticulously recorded. In principle, the VDE operates on a Markovian assumption, which could be theoretically compro- mised when some influential factors remain unobserved. Nevertheless, we maintain confidence in SIDEC's adaptability to such tasks, as it primarily relies on node features, potentially mitigating these concerns.
Thirdly, our evaluation of SIDEC was exclusively conducted on synthetic datasets, leaving a void in the examination its performance real-world, intricate data. This highlights the pressing need for comprehensive and reliable real-world structural inference dataset, an omission in the existing landscape. However, it's important to acknowledge that these limitations serve catalysts for fu- ture research endeavors aimed at enhancing both SIDEC and structural inference methodologies in general. By surmounting these challenges, we can substantially expand the application domains of SIDEC, offering invaluable contributions to researchers across diverse fields.
F BROADER IMPACT
Much like NRI, MPM, ACD, iSIDG, and other structural inference methodologies, SIDEC extends its utility to diverse range of researchers across the realms of physics, chemistry, sociology, and bi- ology. In our investigations, we have demonstrated SIDEC's proficiency in reconstructing expansive graph structures while displaying robustness to variations the dimensionality of node features, underscoring its versatility and broad applicabillity However, the advent of structural inference technology, while profoundly beneficial to many fields, also raises concerns about potential misuse. There exists the possibility of utilizing this technology unveil intimate relationships among users within social networks, which, if misapplied, could encroach upon individuals' privacy.
27


Reviews:
[{"review_id": "oUb0gUFSyh", "reviewer": "Reviewer_Fg5F", "summary": "This paper proposes a new method for structure inference from networked dynamical systems. At a high level, the authors' methodology is as follows. First , they use a variant of VAE's -- called a Variational Dynamics Encoder (VDE) -- to find a low-dimensional embedding of the time series of each node in the network. The network structure between nodes can then be determined by computing the partial correlation scores between the nodes' latent representations. The authors then test their methods on several benchmark datasets. Their methods perform favorably against prior approaches.", "questions": "- (pg. 4) You write that $\\tau$ represents the length of the time window required for the system's dynamics to be considered Markovian. What do you precisely mean by this? Why would the time window affect whether or not the system is Markovian?\n- What is the dimension of the embedding of node-level dynamics produced by the VDE? And what is the effect of the dimension on the performance of the algorithm?\n- It's not clear to me why VDE is used to produce an embedding, rather than another embedding procedure. Was there any particular reason for using this method?\n- What are the \"Springs\" and \"NetSim\" simulations?\n- What does \"level of added Gaussian noise\" mean? Are you changing the mean of the noise? Please be clear.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "The paper is very well written, and the approach is clearly laid out. The goal of the paper -- that of learning the network that facilitates dynamic processes -- is of wide applicability and significant interest. A major strength of this work is that the proposed framework performs better than several other prior approaches for this task.", "weaknesses": "- There are some missing details about key elements of the procedure. How are the dynamics modeled, and how does the adjacency matrix A influence the dynamics? How is A estimated from the embedded representations? It's hinted that this is done by the partial correlations formula, but it doesn't seem to be explicitly stated.\n- It's not intuitive to me why the node-level dynamics can be predicted from a single node's trajectory alone, without needing to know the dynamics of other nodes (which certainly influence the trajectories of the node in question). In other words, won't the accuracy of trajectory predictions be strongly affected if the VDE is applied separately to each node trajectory, rather than jointly? \n- The authors state that scalability is a major contribution of the method. But in the Experiments, the largest network has only 250 nodes, which makes me question whether there is sufficient evidence for scalability. \n- Various missing details in the Experiments section -- see the \"Questions\" section below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lgBMjUWAf2", "reviewer": "Reviewer_5DvY", "summary": "This paper proposes a new model for structure inference in dynamical systems, called SIDEC. \nIt combines two methods, the Variational Dynamics Encoder (VDE) and the Partial Correlation (PCOR), to infer the directed graph structure of a dynamical system from its trajectory data. The method captures the minimum information of node feature dynamics and can effectively reduce feature dimensions. It is applicable to time-series data with multi-dimensional and one-dimensional features. It evaluates SIDEC on 20 datasets and compares it with other benchmark methods. The results show that SIDEC outperforms other methods in terms of average AUROC metric and it is robust to Gaussian noise.", "questions": "1.\tSIDEC includes PCOR, and some PROR models are also mentioned in the Structural inference with correlations and partial correlations section. Why are there no comparative experiments of these models included in the experimental section?\n2.\tJudging from the results of the ablation experiment, the optimal size of the time window is 1. It seems that the existence of the time window does not fully improve the performance of the model. Does this mean that it is not necessary to use VDB?\n3.\tSuggestion: maybe you can adjust the combination of loss to the weighted sum of the reconstruction loss and the autocorrelation loss to adjust the impact of time window，and  monitor the change of the performance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper introduces a novel model, SIDEC, that combines two methods, VDE and PCOR, to infer the directed graph structure of a dynamical system from its trajectory data. \nThe paper provides a comprehensive introduction, background, and motivation for the problem, which make it easier to follow. \nIt seems that SIDEC can capture the minimal information of node feature dynamics and effectively reduce feature dimensions, leverage temporal information to infer directed edges using conditional correlations, and perform robustly against Gaussian noise .\nIt evaluates SIDEC on 20 datasets and compares it with other benchmark methods and the results show that SIDEC outperforms other methods in terms of average AUROC metric.", "weaknesses": "1.\tThis paper mainly designs and analyzes the SIDEC framework based on information bottleneck theory and partial correlation coefficient, but it does not give clear mathematical proofs and theoretical guarantees to explain how they influence each other.\n2.\tExperiment Datasets are all artificially generated and may not fully reflect the complexity and diversity of the real world. Therefore, the experimental results of this article may have certain biases and limitations, and further testing and evaluation on more real data sets are required.\n3.\tThis method combines VDB and PCOR, but the paper does not give the impact on training time after the combination.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IdL02lw9g2", "reviewer": "Reviewer_LxHJ", "summary": "The paper introduces an innovative approach to structural inference (SIDEC), employing dynamics encoding and partial correlation coefficients. The authors employ variational inference to encode node dynamics within latent variables and reorganize these variables temporally to reconstruct directed graph structures. The resulting method stands out for its scalability, accuracy, and versatility, outperforming existing approaches as demonstrated through experimental results. The paper additionally includes a comprehensive implementation appendix and provides a link to the anonymous GitHub repository hosting the SIDEC codebase, ensuring reproducibility.", "questions": "1. The choice of using partial correlation as the final step in reconstructing the graph structure merits further clarification.\n\n2. A suggestion for refining the paper layout is to consider rearranging Algorithms 1 and 2 before presenting Algorithms 3 and 4 in the appendix for better logical flow.", "limitations": "", "rating": 10, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper presents a novel approach to structural inference, combining variational dynamics encoding with partial correlation coefficients, demonstrating superior performance compared to existing methods, as validated by experimental results. The use of a variational model to encode node dynamics addresses scalability challenges from previous methods, and this application of Variational Autoencoders (VAE) in structural inference is a notable contribution.\n\n2. The method exhibits impressive scalability, accommodating trajectories with both one-dimensional and multi-dimensional features, making it applicable to a wide range of scenarios.\n\n3. The paper is exceptionally well-organized and articulated, employing clear and concise language throughout.\n\n4. The authors provide a detailed explanation of their proposed method, including mathematical formulations and implementation details, enhancing its accessibility and reproducibility.\n\n5. The evaluation is meticulously crafted and comprehensive, successfully establishing the superior performance of SIDEC in comparison to existing methods. The additional experimental results presented in the appendix further bolster the credibility and persuasiveness of SIDEC's superiority.\n\n6. The limitations of the proposed method are conscientiously addressed in the appendix.", "weaknesses": "While the paper is commendable overall, one potential weakness lies in the absence of an evaluation on real-world data. The authors acknowledge this in the \"Limitations\" section, citing the challenges of collecting such data. It would be beneficial to include plans for addressing this in future work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)