IPSI: Enhancing Structural Inference with Automatically Learned Structural Priors
Authors: Zhongben Gong
Xiaoqun Wu
Mingyang Zhou
Venue: NeurIPS 2025 poster ()
Keywords: Structural Inference
AI for Science
Graph Neural Networks
Primary area: machine_learning_for_sciences
TL;DR: We propose IPSI, a general structural inference framework that improves performance by injecting pretrained structural priors into variational inference module.

Abstract: We propose IPSI, a general iterative framework for structural inference in interacting dynamical systems. It integrates a pretrained structural estimator and a joint inference module based on the Variational Autoencoder (VAE); these components
are alternately updated to progressively refine the inferred structures. Initially, the
structural estimator is trained on labels from either a meta-dataset or a baseline
model to extract features and generate structural priors, which provide multi-level
guidance for training the joint inference module. In subsequent iterations, pseudolabels from the joint module replace the initial labels. IPSI is compatible with
various VAE-based models. Experiments on synthetic datasets of physical systems
demonstrate that IPSI significantly enhances the performance of structural inference models such as Neural Relational Inference (NRI). Ablation studies reveal
that feature and structural prior inputs to the joint module offer complementary
improvements from representational and generative perspectives.

IPSI: Enhancing Structural Inference with Automatically Learned Structural Priors
Xiaoqun Wu' Mingyang Zhou College of Computer Science and Software Engineering, Shenzhen University 2bgong17290gmail. com, xquu@szu. edu. en, zmy@szu. edu. on
Zhongben Gong
Abstract
We propose IPSI, a general iterative framework for structural inference interact- ing dynamical systems. It integrates a pretrained structural estimator and joint inference module based on the Variational Autoencoder (VAE); these components are alternately updated progressively refine the inferred structures. Initially, the structural estimator is trained on labels from either a meta-dataset or baseline model to extract features and generate structural priors, which provide multi-level guidance for training the joint inference module. In subsequent iterations, pseudo- labels from the joint module replace the initial labels. IPSI is compatible with various VAE-based models. Experiments on synthetic datasets of physical systems demonstrate that IPSI significantly enhances the performance of structural infer- ence models such as Neural Relational Inference (NRI). Ablation studies reveal that feature and structural prior inputs the joint module offer complementary improvements from representational and generative perspectives.
1 Introduction
In many domains, dynamical systems can be understood collections of interacting agents, ranging from physical, biological to multi-agent systems [9] 5 modeled by an interaction graph, where nodes represent agents and edges denote the existence of interactions. Understanding the structure of such a graph essential for analyzing, controlling, and optimizing the behavior of the underlying system. However, in practice, the interaction structure is frequently unobserved or only partially known, and only the observable agent states are available. For instance, in molecular biology, understanding the interactions between drug compounds and target proteins is critical for applications such as drug discovery, side-effect prediction, and drug repurposing [6] These interactions are typically governed by underlying molecular structures and biochemical affinities, which are difficult to deduce purely from theoretical analysis. Moreover, experimentally identifying all potential interactions is often costly and tine-consuming, Structural inference offers a promising alternative by uncovering latent interaction patterns based on molecular dynamics that can be observed more easily, thereby reducing the reliance on expensive experimental procedures and providing mechanistic insights into pharmacological activity [5].
15 [3] [[]. These interactions are often
As milestone in the field of structure inference, the Neural Relational Inference (NRI) model leverages the latent space of a Variational Autoencoder (VAE) 7 to model underlying structures [8]. However, it may face challenges when applied to complex physical systems such charged particle interactions. This is largely due to the limitations of unsupervised joint training and the reliance on overly simplistie priors for latent variables. While some approaches incorporate prior structural knowledge, few systematically address how to conveniently obtain reliable structure priors across diverse dynamical systems.
*Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

To address these challenges, we propose the Iterative Pretrained Structural Inference Framework (IPSI), an iterative framework for structural inference that combines pretrained structural estimator ppror with a VAF-based joint inference module Jajin-- Specifically, Sojoitt receives embedding representations including structural information and learnable structural prior both provided by SI prior- These components, together with state labels, provide multi-level supervision for the various modules within joint during training. Meanwhile, SIjoint and Siriior are alternately updated in an iterative process, allowing the model to escape local optima and progressively refine its structure inference.
Slrioo can be seen structural prior network, and is trained under supervision using structural labels. To this end, we propose two complementary strategies generate structural labels for the first iteration, while in subsequent iterations, pseudo-labels from Sijaini are used instead. The design of IPST follows general paradigm common to many structure inference models, enabling seamless integration with variety of existing frameworks. Experimental results demonstrate that IPSI significantly improves the performance of multiple baselines and achieves state-of-the-an accuracy across several datasets.
Related work
Structural inference aims uncover latent interaction structures from observable sequences agent states. Compared to traditional statistical and information-theoretii methods, recent deep learning partccccessaattiaaarrl those based on neural networks- oofer enhanced capabilities in modeling high-dimensional data requiring fewer assumptions. A major milestone in this direction is NRI, which pioneered the use of VAEs for inferring latent structures [8. NRI employs a fully connected Graph Neural Network (GNN) encoder to propagate information across nodes, while modeling latent variables as probabilistic adjacency matrices. These latent structures are then used by the decoder to predict future states. However, NRI relies oversimplified priors over the latent space and faces challenges in jointly training the encoder and decoder without external supervision -articularly in complex systems.
Building upon this foundation, some subsequent works have explored incorporating prior knowledge of real-world interaction structures into the NRI framework. Li et al. (II and Chen et al. 2 introduced structural priors derived from real-world networks-such as degree distributions, sparsity, and connectivity-as regularization terms to guide latent structure learning. Along with these structural priors, Wang et al. [16] proposed an iterative training scheme which edge weights are refined during training to emphasize likely interactions and suppress noisy connections. While these methods demonstrate improved structure inference, they often depend on manually crafted priors that difficult to generalize across domains.
In addition incorporating structural priors, several variants and extensions of NRI have been proposed to enhance performance and flexibility, including alternative decoding mechanisms 21]. multi-interaction systems (22), and improved optimization strategies 4 among others [1
18 [9]. Despite these advances, limited attention has been given to the question of how to obtain informative structural priors efficiently. This gap becomes critical in unsupervised settings, where the joint learning of dynamics and structure can suffer without effective prior guidance.
Preliminaries
3.1 Notations and Problem Formulation
In this paper, denote the number of agents in the interacting system as N. The state of agent i at time is represented by a vector x! E Rd, where is the dimensionality of the state space and the number of time steps is denoted as T. For simplicity, the collection of all agent states at time í is denoted as x' = [x E Nx the time series data of agent i is denoted as
x; {x E Rxx and the collection of all time series data of all agents denoted as x [x, E NxTxd The underlying interaction structure among the agents is modeled as a graph G = (V,E), where V (u1, .2,.... UN) represents the set of vertices (agents), and C V represents the set of edges encoding the interactions between agents. The interaction graph is characterized by its adjacency tensor E IR X to consider multiple types of interactions,

---

where each entry 2isk is defined as:
1, if there is an interaction of type & from agent to agent j,
Zisk 0, otherwise.
(1)
The problem of structural inference we consider in this paper to recover the hidden interaction graph G or equivalently its adjacency tensor from the observed sequences of agent states x. More formally, given the agent states over time, the objective to learn a function f: x -> z, where f()) captures the underlying structure from states. This problem involves both inferring the structure of the graph G and modeling the interactions mechanisms that govern the dynamics the system.
3.2 Neural Relational Inference
3.2.1 Model Overview
NRI is a foundational method for unsupervised structure learning in dynamical systems, which learns the latent interaction graph among agents from observed trajectories without ground-truth edge information 8
Built the VAE framework, NRI consists of two main components: GNN-based encoder and a
decoder. The encoder processes observable node sequences x E R Txd through message passing extract structural information and outputs distribution over latent edge types, modeled as categorical variables with classes, forming the adjacency tensor z E R NxNXX,
(x), go (z(x)) = softmax(h)
(2)
Since directly sampling the discrete adjacency tensor z is non non- con-differentiable. NRI employs the numbel-Softmai trick [12] for backpropagation. The inferred graph z then fed into the based decoder, which predicts future dynamics via message passing on the sampled interaction graph. The decoder optimized minimize reconstruction loss between predicted and true future states:
zij = softmax ((hij + gg//)), (x(x)
II Po |xl=t 1=1
(3)
The model is trained by maximizing the evidence lower bound (ELBO):
ELBO Ego (z) (z)x) poop))oooooppppppooo (x|2)] (KL (()))))((z))
where the first term minimizes state prediction error and the second regularizes the latent space by constraining ( (z(x) to align with the prior p(z) (typically uniform).
3.2.2 Limitations of NRI
Despite the elegant formulation and general applicability of NRI, suffers from several notable limitations when applied to complex dynamical systems. First, the uniform prior imposed over the latent interaction graph often deviates significantly from the true underlying structure which leads to suboptimal inference. Second, the joint unsupervised training of encoder and decoder relies solely on trajectory reconstruction loss, which may not provide sufficiently informative gradients to uncover accurate interaction structures. Lastly, the absence of external structural supervision makes one-shot training strategies prone to premature convergence to suboptimal solutions. These limitations highlight the need for more flexible and informed structure inference framework one that can incorporate external structural cues to provide multi-level supervision or guidance for both the encoder and decoder, and employ iterative optimization to escape local minima.
4
Model design
4.1
Motivation and Overall Pipeline Architecture
Most VAE-hased structural inference models can be summarized as follows (trajectory embedding is separated from the encoder):
trajectory data
embedding embedding vector
encoder
inferred structure
decoder
predicted trajectory

---

Unifom
Learned
Ground
10
10
(a) Uniform prior
(b) Learned prior
(c) Ground truth
Figure 1: Comparison of two kinds of prior distributions and the ground truth.
This paradigm consists three trainable components: trajectory embedding, encoder, and decoder. However, only the ground-trude trajectories are available as supervision signals. In IPSI, we address this issue by providing pretraining inputs for the trajectory embedding and encoder modules. These two inputs are used to guide the embedding and encoder modules, and together with state labels, achieving simultaneous guidance of three trainable modules, thus effectively alleviating the difficulties caused by long chain joint training. Specifically, the pretrained structure estimator Slprior extracts features from raw trajectories. These features are concatenated with the original node embeddings and fed into the encoder of SIjoint- Since the training of Slprior is supervised by structural labels, this strategy will result richer representations that embed structural cues.
Next, prior predicted edge probabilities define an informed prior (x) replace the standard uniform prior p(z) in the KL divergence term with this learned prior:
(z|x) of structure. We
LK.. = DKI. (go((()
(z)x)).
(5)
While the uniform prior adopted in NRI serves as a regularization mechanism to constrain the structure of the latent space, may significantly deviate from the true underlying distribution. In contrast, IPSI employs learned prior that functions both as regularizer and soft target. This approach not only guides the early stages of training but also mitigates adverse effect of KL divergence loss on prediction accuracy during later training phases. Figure IreeeTshooe shows comparison between two kinds of priors and the ground truth.
A key challenge in the IPSI framework lies in how to obtain structural labels for supervising the training of SIpioor since ground-trui truth structures are unavailable in unsupervised structural inference. In this work, we propose an iterative training framework. In the first round of iteration, the structural labels are derived either from a synthetic meta-datased model, depending whether prior knowledge of system dynamics is available (more details will be introduced in Section 4.5). subsequent rounds of iteration, as SIjoin has been trained and achieves more accurate inference, its outputs are used new pseudo-labels to update SIpriee- Through this iterative process, both SIprior and SI are progressively refined, this alternating update strategy will help the model escape from local optima and ultimately achieve performance that significantly surpasses that the baseline models. Figure 2 shows the complete pipeline architecture of IPSI.
from pseudo-labels inferred by a baseline
Given that IPSI is designed based on the general paradigm of VAE-based structural inference models. possesses universality and can be integrated into various such models. Next, we will use the NRI model as an example to illustrate how IPSI integrates and enhances the performance of the NRI model.
4.2 Pretrained Structure Estimator
The pretrained structural estimator, denoted as SIprior. adopts the same GNN-based encoder architec- ture as the joint inference module SIjoine, but is trained in supervised manner to extract trajectory embedding containing more structural information and generate learnable structural priors, and then input them into the joint training model SIjoint. The encoder comprises temporal node embedding module followed by a fully connected message-passing network. To better capture the temporal dynamics of each agent's 's trajectory, we replace the Multi-Layer Perceptron (MLP) embedding used

---

Sl Sljoint
Trajectories embedding
GRU
Latent Space
GNN Decoder v-
States Prediction
V--8
e--V
Concatenation
Regularization
Supervision
Synthatic meta-dataset
Siprior
supai
Trajectories
GNN Encoder
Latent Space
Subsequent supervision
GRU
V
Supervision [frrst round)
embedding
Baselline model
Pretrained Imputs
Figure 2: Illustration of the iterative supervision pipeline. The first iteration uses ground-truth labels of meta-dataset or pseudo-labels from baseline model to supervise the training of Slprior which then produces an informative prior to guide ST Jointt The process is repeated in a loop to progressively improve both modules.
in NRI with Gated Recurrent Unit (GRU)
33:
Node Embedding:
h Iaprior Gx,((x),
(6)
m-re: h 1.prior (1.))
fprlor (h .prior h (.proor)).
- v: h 2.ptior pprlor
Σ jaL
h I,prior
(8)
(b.))
+ h 2.prior
frrior
(h, 2.prio
. h2piiio
(9)
(3.3)
The final edge representation 2.prio is then used to predict the edge type through categorical (1.5)
posterior distribution:
prior d
(z)f(x) = softmax (n 212.ptirr)
(i.d)
(10)
where á denotes the learnable parameters of the encoder, and x represents the full input trajectories. We use ground-truth labels aiyk corresponding to K types of edges to supervise the prediction of edge existence via the binary cross- cooss-nntoppy loss:
Cprior
prior Oajk log de
(Zigh = 1/x).
(11)
K A=1
Details of the source of the structural supervision signal 0izs will be introduced in Section 4.5
4.3 VAE-based Structural Inference Model
The VAE-based structural inference model Sljoiut adopts an encoder-decoder architecture similar to the NRI framework, but introduces key enhancement by incorporating prior knowledge from SIprior- Specifically, SIjoint encoder concatenates its own node embeddings with those provided by SIprior along the feature dimension. The GRUs in Eq. (6) and Eq. (12) and the learnable prior module Eq. (18) are independently parameterized, as SIjoint and Siprior serve distinct roles: the former includes both encoder and decoder for inference and reconstruction, while the latter contains only an encoder that generates structural priors to guide iterative updates. Except for concatenating trajectory embeddings from SIprior after the trajectory embedding layer, the decoder and encoder of
5

---

SIjoint are identical to the baseline model (such as NRI):
Node Embedding: h = [G) hj h...riorr.
(12)
is worth noting that this modification is relatively minor but both effective and widely applicable. Since encoding raw trajectory data into latent node representations is highly generic step, this design enables IPSI to serve as a general-purpose structural inference enhancement framework that can be flexibly combined with various existing encoder-decoder architectures.
4.4 Training with Hybrid Loss
As introduced Section 4.1 we replace the simple uniform prior p(z) ) NRI loss with a learnale prior o0 (z]x), which serves both as regularizer and as soft target. This learnable prior demon- strates significant advantages over the uniform prior, and, compared to manually crafted structural prior losses proposed in various subsequent works such sparsity and smoothness, it requires task-specific design and avoids the difficulty of tuning numerous additional hyperparameters:
L = Cp DKL (go(z)x) g.
prior (z)x)),
(13)
where I controls the influence of the structural prior. Similar to the modifications made in the model, this modification is lightweight yet effective, and can readily incorporated into most VAE-based structural inference models. As it requires no modification to the base architecture and simply replaces the prior distribution.
4.5 Two Sources of Supervision for Pretraining
As introduced in Section 4.1 in order to train SIprior with structural labels without ground-truth structure of the target system, we introduce two complementary strategies that provide supervision signals under different assumptions about prior knowledge.
4.5.1 Supervision from Synthetic Meta-Dataset with Prior Knowledge
When partial prior knowledge of the system's 's interaction form is available, we propose constructing a synthetic meta-dataset of labeled interactions. This dataset includes multiple simulated systems with similar dynamical properties but excludes the exact target configuration, aiming to enable SIprior to learn transferable structural motifs applicable to unseen yet related systems.
For example, in systems governed by radial forces (c.g., springs or charged particles systems), generate synthetic trajectories from various parameterized systems with edge types such as attractive, repulsive, or null, and with different distance dependencies (e.g., constant, inverse, inverse-square). Importantly, we exclude systems matching the specific parameters of the target system, thereby simulating the following situation: "The observer aware that interactions are radial nature, but has knowledge of the proportion of attractive, repulsive, null connections, nor the precise functional relationship between force magnitude and distance."
As introduced in Section 1.1 Siprior is trained on the synthetic meta-dataset using ground-truv -tuth structural labels as supervision the first round of iteration, and trained on the target dataset using pseudo-labels as supervision in the following rounds.
4.5.2 Supervision via Pseudo-Labels without Prior Knowledge
When prior knowledge of the system dynamics is unavailable, we first train baseline model such as NRI and use its structure inference outputs as pseudo-labels to provide supervision. Although this approach does not incorporate the additional information from the synthetic meta-dataset, the pseudo-labels still offer more informative prior estimate than uniform prior. Furthermore, the subsequent iterative process described in Section is applied in the same manner, alternating the optimization of modules S1 Slprior and SIjoint, ultimately lcading to improved performance.

---

Experiments
5.1 Datasets
To evaluate the effectiveness of the two supervision strategies, we focus on the three synthetic physical systems proposed in -8]the spring, charged particle, and Kuramoto oscillator systems-as synthetic meta-datasets are constructed based on the spring and charged particle systems. We adopt the same simulation configuration as in the original setup: each system is simulated for 5000 time steps, with or 10 interacting objects, and the interaction graph is set to undirected. The data is split into training, validation, and test with 5:1:1 ratio. And the details about the synthetic meta-dataset are described in supplementary materials.
5.2 Baselines
We compare our model with several recent structural inference approaches that focus on physical systems with available and complete codes:
NRI 8
a VAE-based structural inference model that jointly learns the relations and dynamics.
SUGAR
method that introduces structural prior knowledge for structural inference.
MPM 2 method that combines a relation interaction mechanism and apatio-temporee message
passing mechanism.
iSIDG [6] : method that iteratively updates the encoder structure based the inferred structure.
Our model is evaluated both the with prior (w/ prior) and without prior (w/o prior) situations, which correspond to different supervision methods described in Section 4.5 And the evaluation metric edge classification accuracy, which reflects how accurately the inferred edge types match the ground truth. Due to space limitations, the results on trajectory prediction error are provided in the supplementary materials. We apply IPSI framework on two different models, NRI and MPM, and report the improved performance to evaluate the generality and flexibility of IPSI.
5.3 Results
The experimental results are summarized in Table Across three types of physical systems, the performance of models enhanced with IPSI is significantly better than the original model. This improvement is particularly pronounced on the Charged Partiele dataset, where the interactions include both attraction and repulsion, leading to more complex motion patterns.
Table 1 also presents the performance of SIprior trained exclusively the synthetic meta-dataset. While SI Sp outperforms the NRI baseline on charged particle datasets, it is still surpassed by the full IPSI pipeline, indicating that SIjoint actively refines the initial priors through joint training rather than merely replicating them. Moreover, on datasets with lower baseline accuracy, IPSI performs better when w/ prior, highlighting the complementary benefits of the two supervision strategies. Finally, despite the non-radial nature of interactions in the Kuramoto system, SIprior still provides reasonable structural estimate, demonstrating its generalization capability. This suggests that constructing larger synthetic meta-dataset to train a more powerful Sl innspied by the scaling principles of large language models could be a promising direction for future work.
5.4
Additional Evaluation the DoSI Benchmark
To further assess the gencralization ability our approach under different graph structures, we conducted additional experiments on the DoSI benchmark (20), which simulates dynamical trajec- tories over empirically-deriiedd graphs from real-world domains. particular, we evaluated three biologically inspired datasets, each containing 15 nodes: Brain Networks (BN), Gene Regulatory Nenworks (GRN), and Vascular Networks (VN). Besides the conventional Springs simulator, we additionally used the NetSims simulator, which models brain activity by assigning nodes to brain regions and edges to their interactions [4]]
For this study, applied our method under the IPSI (w/o prior) configuration. Table reports the AUROC scores on all six datasets. We observe that IPSI consistently outperforms the baseline methods (NRI (8), MPM
2 across both simulators, achieving state-of-the-ar performance among
7

---

Table 1: Edge prediction accuracy (%) on Springs, Charged, and Kuramoto systems. For NRI, SUGAR, and MPM. we directly used the results from [8] and 2 after checking the consistency of the benchmark, For iSIDG [16], we used our own measured results because different datasets generation methods were used.
Model
Springs
Charged
Kuramoto
objects
NRI
99.9 + 0.0
82.1 0.6
96.0
SUGAR
99.9 + 0.0
82.9 + 0.8
91.8 +
MPM
99.9 + 0.0
93.3 + 0.5
97.3
SSIDG
99.9 = 0.0
9.67
9.3
IPSI (NRI-based, w/ prior)
99.9 = 0.0
9..2
97.2 0.3
IPSI (NRI-based, w/o prior)
99.9 + 0.0
89.3
97.4 0.3
IPSI (MPM-based, w/ prior)
99.9 + 0.0
94.3 0.4
98.0 1 0.3
IPSI (MPM-basedd w/o prior)
99.9 = 0.0
94.3 = 0.3
98.1 = 0.2
Slprior (NRI-based, w/ prior)
89.7 + 0.2
86.7 + 0.3
77.1 0.5
Supervised
99.9 = 0.0
95.4 0.1
99.3 = 0.0
10 objects
NRI
98.4 = 0.0
7.4
7.3
SUGAR
98.3 = 0.0
7,9
14.0
MPM
99.1 0.0
0.2
8.6
SIIDG
99.2 + 0.0
0.3
0.5
IPSI (NRI-based, w/ prior)
99.2 + 0.1
77.1
77.3 +0.4
IPSI (NRI-based, w/o prior)
99.2 + 0.1
75.3 +33
76.9 +0.3
IPSI (MPMM-ased, w/ prior)
99.6 = 0.1
10.7
82.6 10.8
IPSI (MPM-based, w/o prior)
99.6 + 0.1
86.2 +0.5
82.8 10.6
Siprior (NRI-based, w/ prior)
87.6 + 0.4
76.2 + 0.5
73.3 + 0.6
Supervised
99.4 = 0.0
89.7 + 0.1
94.3 + 0.8
VAE-based structural inference models. These results suggest that IPSI remains robust in a more realistic evaluation environment.
5.5 Training Dynamics
To further elucidate the training dynamics, track three metrics during single run of STjoint training: the state-prediction mean squared error (MSE), the KL-divergence regularization term, As illustrated in Figure 3al the KL
and the edge-prediction accuracy on the training that the model closely follows the pretrained prior-then rises as SIJoint diverges learn more precise structures. In contrast, both MSE and edge accuracy improve steadily throughout training. This behavior suggests that the model first undergoes a phase of prior imitation and subsequently refines the structures based on the initial priors, resulting in enhanced predictive performance. There three experiments are conducted on charged particle dataset with objects and w/o prior situation.
initially intrrasss-innicalle
5.6 Robustness
Figure 3b ibillustrates the impact of SIprior accuracy and iteration round Sl performance. The results show that even when SIprior is completely untrained (guessing randomly with two edge types will achieve about 50% accuracy), SIjoint still achieves the same performance as the NRI baseline. This demonstrates the robustness of the IPSI framework: even when the prior inputs are poor, IPSI is capable learning to shield against these erroneous priors, maintaining performance comparable to the original baseline model. Meanwhile, the inserted figure shows that the accuracy steadily increases and eventually converges, demonstrating the effectiveness and stability of the proposed iterative training strategy.

---

Table Results on the DoSI benchmark (AUROC, %). All datasets contain 15 nodes. IPSI (w/o prior) maintains strong performance and achieves the best results among VAE-based approaches. Higher better.
Springs (AUROC)
NetSims (AUROC)
Methods
BN
GRN
VN
BN
GRN
VN
NRI
8
99.75
90.55
92.68
99.79
78.08
89.13
ACD
99.75
91.10
94.34
99.87
80.18
80.32
MPM 2
99,98
94.02
96.56
99.95
76.06
91.18
iSIDG
99.97
92.91
96.59
99.91
71.11
91.20
RCSI
99.81
93.01
97.03
99.72
77.45
91.53
IPSI (NRI-based, w/o prior)
99.75
91.19
93.77
99.80
84.40
93.75
IPSI (MPM-based, w/o prior)
99.98
98.95
99.39
99.97
95.32
96.75
130.
TeaT MOUIN
Van
ralia Dvarcace
(a)
(b)
(c)
Figure 3: Three additional experiments conducted on the charged particle dataset with objects (a) Training dynamics of Sljoint. (b) Impacts of iteration round and Siprior accuracy on Sljoint accuracy. (c) Loss perturbation analysis of NRI and IPSI. Each curve shows the loss change under random parameter perturbations, where e denoting perturbation magnitude (x-axis).
5.7 Perturbation Analysis
To gain deeper insight into IPSI, we performed random- perturbation analysis: For both trained NRI and IPSI models, all model parameters were flattened into a single vector, a random unit vector identical dimension was sampled as a perturbation direction, and scaled perturbations were added to evaluate the change in loss. We analyzed two loss functions: (I) the training loss (ELBO objective), and (2) the target loss (I-ACC), reflecting structural inference quality. Across 100 random directions, we computed two quantitative measures: non-optimal proportion (the fraction of perturbation directions yielding lower loss than the original model, within a tolerance) and relative improvement (the mean percentage decrease of loss among those non-optimal directions).
Experiments the GRN dataset (Springs, 15 nodes, w/o prior) yielded the results summarized in Table 3 Under the training loss, both NRI and IPSI exhibit comparable local optimality, suggesting that the Adam optimizer effectively finds stable basins for both. However, under the target loss, IPSI demonstrates more stable minima, reflected by lower non-optimal proportion and smaller relative improvement. The perturbation line plots in Figure TigureBcfurther further visualize these differences. where each curve corresponds to random perturbation direction. For target loss sensitivity of IPSI (bottom right), most curves are monotonically increasing, while target loss sensitivity of NRI (bottom left) shows irregular patterns with many curves decreasing under perturbation. These observations suggest that both models converge to local optima under the training loss, but IPSI converges to better optima under the task-relevant target loss. IPSI effectively reshapes the loss landscape through iteratively updates trajectory embeddings and structural priors, thereby aligning the training objective with the structural inference target.

---

Table 3: Perturbation analysis on GRN (Springs, 15 nodes).
Training Loss
Target Loss
Model
Non-opt. (%)
Rel. Imp. (%)
Non-opt. (%)
Rel. Imp. (%)
NRI
1.16
63
31.7
IPSI (w/o prior)
1.49
6
11.2
5.8
Ablation Study
To analyze the contribution of each pretrained input component, perform ablation study on its two injection points within Sljoint (1) the embedding vector and (2) the structural prior. Three reduced variants are evaluated: one using only the embedding vector, one using only the structural prior, and one using neither-equivalent to an NRI variant with a GRU embedding scheme. Experiments are conducted under the w/ prior setting on the charged particle dataset with five objects, and results are shown in Table 4
Both single-input variants outperform the baseline without pretrained inputs, demonstrating that each component independently improves performance. The full model, combining both the embedding vector and the structural prior, achieves the best accuracy, indicating their complementarity. These findings are consistent with our design rationale: the embedding vector provides enriched represen- tations informed by global structural cues, while the latent prior serves regularization signal that steers the latent distribution toward plausible interaction patterns. Together, they enhance the structural inference both the representation and generative levels.
Table 4: Ablation results on edge accuracy (%) for the Charged dataset with 5 objects.
Model Variant
Charged Particle System
No pretrained input (baseline) Only embedding vector
82.2 + 0.4
89.5 0.7
Only structural prior
84.1 = 0.4
Full model (both inputs)
91.2 0.4
Conclusion and Limitations
In this work, we introduced IPSI, an iterative pretrained structural inference framework that alternates between data-driven inference and atructure-guiee refinement. Through extensive experiments on both synthetic and empirically derived benchmarks, IPSI consistently improved structural accuracy and generalization over existing VAE-based methods. The perturbation analysis further demonstrated that IPSI enhances the aligmment between training and task objectives by iteratively reshaping the loss landscape, providing an intuitive explanation for its superior performance.
key limitation of IPSI lies in the increased computational overhead introduced by its iterative process, which makes it challenging to evaluate on large-scale systems (involving dozens more agents) and real-world datasets. Generalizing structural inference to large graphs remains a major challenge, primarily due the quadratic growth of computational cost with respect the number of nodes when employing fully connected GNN encoders.
Nevertheless, IPSI offers promising direction to address this issue. By leveraging the pretrained structural prior to guide the construction of encoder connectivity-ratte than using it solely as a latent prior-the effective edge complexity can be reduced from O(N2) to ((N) for systems with sparse underlying structures. In future work, we plan to explore scalable variants of IPSI and apply it large-scale, real-world dynamical systems such neural and biological networks, as well as further investigate theoretical properties of its iterative optimization process.
Acknowledgments
Code is http://hhhhhbcccomm////aaa//////
10

---

References
[1] G. Brasó and L. Leal-Taixé. Learning a neural solver for multiple object tracking. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 6247-6257, 2020.
[2] S. Chen, J. Wang, and G. Li. Neural relational inference with efficient message passing mechanisms. In Proceedings of the AAAI Conference Artificial Intelligence, volume 35, pages 7055-7063, 2021.
[3] K. Cho, B. van Merrëenboer, C. Gulcehre, D. Bahdanau, F. Bougares, Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoded for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724-1734. Association for Computational Linguistics, 2014.
[4] A. Comas, Y. Du, C. F. Lopez, S. Ghimire, M. Sznaier, J. B. Tenenbaum, and O. Camps. Inferring relational potentials in interacting systems. In International Conference on Machine Learning, pages 6364-6383. PMLR, 2023.
[5] S. Ha and H. Jeong. Unraveling hidden interactions in complex systems with deep learning. Scientific Reports, 1((1) 2021.
[6] A. L.. Hopkins and C. R. Groom. The druggable genome. Nature Reviews Drug Discovery, 199):727-730, 2002.
[7] D. Kingma, M. Welling, et al. Auto-encoding variational bayes, 2013.
[8] T. Kipf, E. Fetaya, K.-C. Wang, M. Welling, and R. Zemel. Neural relational inference for interacting systems. In International conference on machine learning, pages 2688-2697, Pmlr, 2018.
[9] Kwapień and S. Droidž. Physical approach complex systems. Physics Reports, 515(3- 4):115-226, 2012.
[10] J. Li, H. Ma, Z. Zhang, J. Li, and M. Tomizuka. Spatio-temporal graph dual-attention network for multi-agent prediction and tracking. IEEE Transactions on Intelligent Transportation Systems, 23(8):0056611566 2021.
[11] Y. Li, C. Meng, C. Shahabi, and Y. Liu. Structure-informed graph auto-encoder for relational inference and simulation. In ICML Workshop on Learning and Reasoning with Graph-Structurer Data, volume 8, page 2, 2019.
[12] C. J. Maddison, A. Mnih, and Y. W. Teh. The concrete distribution: A continuous relaxation of discrete random variables. In International Conference on Learning Representations, 2017.
[13] A. Pratapa, P. Jalihal, J. N. Law, A. Bharadwaj, and T. Murali. Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nature methods, 17(2):147-154, 2020.
[14] S. M. Smith, L. Miller, G. Salimi-Khooshich, M. Webster, C. Beckmann. T. E. Nichols, J. D. Ramsey, and M. W. Woolrich. Network modelling methods for fmri. Neuroimage, 54(2):875 891, 2011.
[15] M. Tsubaki, K. Tomii, and Sese. Compound-protein interaction prediction with end-to-end learning of neural networks for graphs and sequences. Bioinformatics, 35(2):309-318, 2019.
[16] A. Wang and J. Pang. Iterative structural inference of directed graphs. Advances in Neural Information Processing Systems, 35:8717-8730, 2022.
[17] A. Wang and J. Pang. Active learning based structural inference. International Conference PMLR, 2023.
on Machine Learning, pages
[18] A. Wang and J. Pang. Structural inference of dynamical systems with conjoined state space models. In The Thirth-eighth Annual Conference on Neural Information Processing Systems, 2024.
11

---

[19] A. Wang and J. Pang. Structural inference with dynamics encoding and partial correlation coeffi- cients. In 12th International Conference on Learning Representations (ICLR'24) OpenReview. net, 2024.
[20] A. Wang, T. P. Tong, A. Mizera, and J. Pang. Benchmarking structural inference methods for interacting dynamical systems with synthetic data, Advances in Neural Information Processing 2024.
Systems, 77:135129-135115
[21] A. Wang, T. P. Tong, and J. Pang. Effective and efficient structural inference with reservoir computing. In International Conference on Machine Learning, pages 36391-36410. PMLR, 2023.
[22] E. Webb, Day, H. Andres-Terre, and Lió. Factorised neural relational inference for multi-interaction systems. arXiv preprint arXiv:1905.08722 2019.
[23] Y. Yang, B. Feng, K. Wang, N. E. Leonard, A. B. Dieng, and C. Allnc-Blachhette Behavior- inspired neural networks for relational inference. arXiv preprint arXiv:2406. 14746, 2024.
[24] Zheng, Z. Li, K. Fujiwara, and G. Tanaka. Diffusion model for relational inference. arXiv preprint arXiv:2401.16755, 2024.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state the main contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Due to page limitation, we describe the limitation of our work with a few words in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
The paper does not include any theoretical theorems or formal proofs; it focuses on algorithmic design and empirical validation.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All necessary details for reproducing the results are provided in the paper and supplementary materials, including datasets, simulation settings, and training protocols. An anonymized GitHub repository has been prepared and will include complete code and instructions.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
14

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We provide anonymized public GitHub repository at https://githh com/ anon pwte-otsebbo-ooeee -eiiiissi-22////// 20125/prbbotsi--ee anoo The full code and instructions will be added. No author-identifying information included.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
mtiiss.cc/public//ee
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section and supplementary materials detail the the training and test details. These details will also be available in the anonymized repository.
and
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All quantitative results are reported with standard deviations over multiple runs, indicating statistical robustness of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: The details of the computational resources used the experiment are described in the supplementary materials
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:///wwwarpppcccc////////aiaa
Answer: [Yes]
The work complies with the NeurlPS Code of Ethics. It uses synthetic data, does not involve human subjects, and poses no foreseeable risk of harm, privacy violations, misuse.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper focuses on a structural inference framework for physical simulation systems with no immediate real-world deployment application risks.
16

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The work poses no high-risk misuse scenarios.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets codes and models used are publicly available and properly cited. Their use complies with the original licenses.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
17

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ com/dataseta has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: This paper proposes new structural inference framework. Documentation on it will be provided in the anonymized GitHub repository.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research does not involve crowdsourcing any form of human subject experimentation.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No IRB approval is required as no human subjects are involved.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
18

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: No LLMss were used for core method development. LLMss have been used for general writing assistance only.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neuri)) hatt::///earrrppcccoooooerrreeee cc///oofeeeenees/2225//.... for what should or should not be described.
19


Reviews:
[{"review_id": "GJbijVNmOT", "reviewer": "Reviewer_FSLU", "summary": "This paper aims to identify the internal relational structure of dynamical systems and predict their evolution in time. The authors use a graph-based architecture to infer a variational latent space inspired by VAEs which encodes those relations from trajectory data, in a two stage procedure. First, a prior is trained based on labelled latent data or pseudo-labels. Then, a second network is trained based on the structural information learnt by the prior network. This provides a significant advantage over only training in a single step without latent labels, as previous works (NRI). The method is tested over multiple trajectories in three datasets (spring-like dynamics, charged particles and Kuramoto system) showing an improvement over the baselines.", "questions": "* The prior distributions scales with O(N^2). The paper only explores two size of systems: 5 and 10 bodies. How scalable is this approach to bigger systems and sparser interactions? For example, a long chain with many bodies in it. Do the authors have an intuition or maybe have tested this method in bigger settings?\n* Have the authors tried to use an MLP instead of a GRU for the node embeddings, as in the original NRI paper? Is this an important architectural choice? \n* What if the pseudo-labels are poorly estimated? Then the joint network would have bad labels and might result in a decrease in accuracy. How difficult is to train this pipeline?\n* Line 131: \"...from raw trajectories, These features...\" might refer to \"...from raw trajectories. These features...\".\n* Equation 5: Closed parenthesis is missing.", "limitations": "Limitations are addressed appropriately.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n* The technique is valid with both labelled and unlabelled data in the latent space.\n* The method is tested over 4 different baselines and ablation studies.\n* The code is provided (not publicly available yet).\n\nWeaknesses\n* The results are modest with respect to the base architecture (MPM).\n* Usually in real world applications the latent space labels are unknown, and in that case (supervision via pseudo-labels) the method is quite pointless even though the results are marginally better than the vanilla architecture.\n* There is no mention about the computational overhead of the iterative process and the addition of an extra network.\n* The study is restricted to small systems with constant edge interactions ($\\sigma$) and fixed graph topology.", "was_revised": true, "final_justification": "My main concerns were the limited applicability to toy datasets and arbitrary architectural choice, and those were satisfactorily addressed in the rebuttal with two additional experiments (DoSI benchmark dataset and MLP ablation study). The authors also provided the computational time overhead with respect to the baseline method, and solved all the minor issues pointed out. Based on the author's rebuttal, I've decided to rise the initial score to an accept.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive feedback. Below, we address each point raised and provide clarifications and additional evidence where needed.\n\n> **W1**: The results are modest with respect to the base architecture (MPM).\n\nMPM is a strong baseline on the synthetic physical simulation dataset. While the improvements achieved by the MPM-based PreBoot-SI appear small in absolute terms (e.g., from 93.3\\% to 94.3\\% on the charged dataset), we would like to point out that such improvements are meaningful in context: this corresponds to a 15% relative reduction in error rate (from 6.7% to 5.7%). Moreover, for structural inference tasks in complex situations, perfect recovery is often unattainable, so even marginal accuracy gains could be meaningful.\n\nIn addition, we conducted supplementary experiments on the benchmark DoSI [1], and on three dataset with real-world structure in DoSI: Brain Networks (BN), Gene Regulatory Networks (GRN) and Vascular Networks (VN). The performance improvement of PreBoot-SI compared to the MPM baseline was more significant. The following table summarizes the improvement on DoSI benchmark, each dataset's interaction graph contains 15 nodes, and the performance metric is the AUROC value:\n\n|                 | Springs BN | Springs GRN | Springs VN | NetSims BN | NetSims GRN | NetSims VN |\n|-------------------------|------------|-------------|------------|------------|-------------|------------|\n| MPM                     | **99.98**  | 94.02       | 96.56      | 99.95      | 76.06       | 91.18      |\n| PreBoot-SI (MPM-based)  | **99.98**  | **98.95**   | **99.39**  | **99.97**  | **95.32**   | **96.75**  |\n\nDoSI also includes results of other VAE-based structural inference models. PreBoot-SI based on MPM has achieved state-of-the-art results on these three datasets. Due to space limitations, we kindly refer the reviewer to our response to Reviewer YANt, 6Y2C or vjqL for detailed results and discussion.\n\n[1] Benchmarking Structural Inference Methods for Interacting Dynamical Systems with Synthetic Data. In NeurIPS Datasets and Benchmarks Track, 2024.\n\n> **W2**: Usually in real world applications the latent space labels are unknown\n\nSimilar to NRI [1], we also target situations where the real structure is unknown. To address this, we propose two complementary methods to obtain pseudo labels instead of directly using ground-truth structure labels:\n\n(a) With prior knowledge of dynamics: We simulate a synthetic meta-dataset comprising systems with similar dynamics (e.g., radial force interactions) but varying parameters. These synthetic systems are generated with structural labels, and the training on them is intended to capture common structure–trajectory patterns shared across systems with similar dynamics.\n\n(b) Without prior knowledge of dynamics: We use a baseline model (e.g., NRI) to generate initial pseudo-labels, which may be inaccurate but still serve as informative priors. Training the baseline model also does not require ground-truth structure labels.\n\nIn both cases, PreBoot-SI progressively refines the structural estimates over iterations.\n\n[1] Neural Relational Inference for Interacting Systems, In ICML, 2018.\n\n> **W3**: There is no mention about the computational overhead of the iterative process and the addition of an extra network.\n\nWe acknowledge that iterative training introduces extra computation, which mainly derives from iterative training of $\\text{SI}_{\\text{joint}}$. The following table shows the training time on Charged Particle dataset on a single 3090 Ti GPU (the complete computational cost will be added in the supplementary materials). PreBoot-SI requires 2–3 times more training time compared to the baseline model and remains within practical limits. Additionally, as mentioned in Section 6, we plan to explore sparsity-guided encoders that potentially reduce complexity from $\\mathcal{O}(N^2)$ to $\\mathcal{O}(N)$. \n\n|   | $\\text{SI}_{\\text{prior}}$|  $\\text{SI}_{\\text{joint}}$ | PreBoot-SI | NRI baseline |\n|------------|---------------------|---------------------|-------------|---------------|\n| 5 Objects  | 5 min               | 1h                  | 6h          | 2.5h          |\n| 10 Objects | 15 min              | 5h                  | 22h         | 10h           |\n\n> **W4 & Q1**: The study is restricted to small systems with constant edge interactions $\\sigma$ and fixed graph topology. How scalable is this approach to bigger systems and sparser interactions? For example, a long chain with many bodies in it. Do the authors have an intuition or maybe have tested this method in bigger settings?\n\nWe agree that evaluating only 5- and 10-node systems with fixed topologies and interaction strength $\\sigma$ is a limitation. In this work, our primary goal is to introduce a general-purpose performance enhancement framework for structural inference, rather than to target specific scalability improvements. To ensure fair and direct comparison with prior methods, we adopted standard benchmark settings from previous work ([1], [2]). \n\nMeanwhile, the primary scalability bottleneck lies in the fully connected GNN encoder employed by NRI and MPM, not in the PreBoot-SI framework itself. As noted in Section 6, our method is compatible with sparse or dynamic encoders, and PreBoot-SI has the potential to utilize learnable structural priors to initialize sparse GNN encoders. In future work, we aim to integrate PreBoot-SI into constructing sparse GNN graphs, enabling inference on longer chains or large-scale systems.\n\n[1] Neural Relational Inference for Interacting Systems, In ICML, 2018.\n\n[2] Neural Relational Inference with Efficient Message Passing Mechanisms, In AAAI, 2021.\n\n> **Q2**: Have the authors tried to use an MLP instead of a GRU for the node embeddings, as in the original NRI paper? Is this an important architectural choice?\n\nWe chose to use GRU instead of MLP for node embeddings, particularly in the $\\text{SI}_{\\text{prior}}$ module trained on the synthetic meta-dataset. This decision is motivated by the nature of the pretraining task: the meta-dataset is designed to capture recurring trajectory-structure patterns across systems with similar dynamics. For example, if two objects consistently move away from each other across time, this may suggest a repulsive edge, which is a temporal pattern that GRUs are well-suited to capture.\n\nTo isolate the architectural effect, we also replaced the MLP-based embedding in the original NRI baseline with a GRU. This change resulted in only negligible performance differences, suggesting that the gains from PreBoot-SI arise primarily from the overall mechanism rather than the choice of node embedding. A brief comparison is provided below, it demonstrates the performance changes of NRI-based PreBoot-SI when replacing node embeddings on Charged dataset (5 nodes), as analyzed above, the replacement of GRU embedding mainly affects the situation with dynamic prior where synthetic meta-dataset is used.\n\n| Model Type                          | Edge prediction accuracy (%) |\n|------------------------------------|-------------------------------|\n| NRI (GRU)                           | 81.9                          |\n| NRI (MLP)                           | 82.2                          |\n| PreBoot-SI (w/o prior, GRU)         | 89.3                          |\n| PreBoot-SI (w/o prior, MLP)         | 88.9                          |\n| PreBoot-SI (w/ prior, GRU)        | **91.2**                         |\n| PreBoot-SI (w/ prior, MLP)        | 89.1                          |\n\n> **Q3**: What if the pseudo-labels are poorly estimated? Then the joint network would have bad labels and might result in a decrease in accuracy. How difficult is to train this pipeline?\n\nAs shown in Figure 3(b), PreBoot-SI remains robust even when initialized with extremely poor pseudo-labels. For example, when we provide randomly generated labels with 50\\% accuracy (i.e., equivalent to random guessing for two edge types), the model still converges to the performance ceiling of NRI baseline.\n\nThis suggests that the bootstrapping process can effectively correct noisy supervision, and that training remains stable and straightforward even when pseudo-labels are poorly estimated.\n\n> **Q4, 5**: Line 131: \"...from raw trajectories, These features...\" might refer to \"...from raw trajectories. These features...\".\nEquation 5: Closed parenthesis is missing.\n\nThank you for pointing them out. We will correct:\n\nLine 131: \"...from raw trajectories, These features...\" $\\rightarrow$ \"...from raw trajectories. These features...\".\n\nEquation (5): we will add the missing closing parenthesis.\n\nOnce again, we thank the reviewer for the valuable comments. We believe these clarifications and additional results strengthen the case for our paper and demonstrate the practical potential of PreBoot-SI."}, {"review_id": "ufyurK7voY", "reviewer": "Reviewer_YANt", "summary": "The paper proposes PreBoot-SI, a bootstrapped framework for structural inference in interacting dynamical systems. The method integrates a pretrained structural estimator and a VAE-based joint inference module, which are alternately updated to refine inferred interaction graphs. The framework improves performance over existing methods (e.g., NRI, MPM) on synthetic physical systems like charged particles and springs, with ablation studies demonstrating the complementary benefits of structural priors and learned embeddings.", "questions": "See the Weaknesses part above.", "limitations": "The authors discuss the limitations of this work.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- Structural inference is a foundational challenge in domains like molecular biology and physics.\n- The paper is well-structured and easy to follow.\n\n## Weaknesses\n- The novelty of this work is limited: The core idea of combining pretrained priors with iterative refinement is a methodological improvement rather than a conceptual breakthrough. Similar strategies (e.g., self-training, pseudo-labeling) are not brand-new in the literature [1]. \n- Narrow evaluation scope: Results focus on synthetic physics datasets (e.g., Kuramoto oscillators), with no validation on real-world biological or multi-agent systems. This limits claims about practical utility.\n- The scalability is concerned: The framework’s computational overhead (e.g., iterative training) and quadratic complexity (due to fully connected GNNs) hinder applicability to large-scale systems, as acknowledged in Section 6. For real-world large-scale dynamical systems, the scalability of the proposed approach is of concern.\n\n[1] Iterative Structural Inference of Directed Graphs. NeurIPS 2022.", "was_revised": true, "final_justification": "Overall, I recommend acceptance.", "rebuttal": "> **W1**: The novelty of this work is limited: The core idea of combining pretrained priors with iterative refinement is a methodological improvement rather than a conceptual breakthrough. Similar strategies (e.g., self-training, pseudo-labeling) are not brand-new in the literature [1].\n\nWe understand the reviewer’s expectation for conceptual breakthroughs. We would like to clarify that our work targets two long-standing pain points in VAE-based structural inference models: (1) **Difficulties in long-chain joint training** under VAE paradigm and (2) **inconvenience and generalization issues** of manually crafted structural priors.\n\nPreBoot-SI does not treat pseudo-labeling or iterative self-training as innovation itself, but rather as a universal reinforcement framework that utilizes **learnable priors** to provide **multi-level guidance** for long-chain joint training. The application of learnable priors and multi-level guidance in this context is novel within the field of structural inference. Pseudo-labeling and iterative self training are methods used to implement these concepts and ensure training stability.\n\nMeanwhile, we sincerely thank the reviewer for highlighting the connection to iSIDG [1]. We offer a detailed comparison below to clarify the conceptual and methodological differences:\n\nModel architecture: iSIDG iteratively **prunes the message-passing structure of the encoder** based on current structural estimates, effectively removing potentially irrelevant edges. In contrast, PreBoot-SI refines a **separate learnable structural prior network** that guides both trajectory embedding and edge inference across training stages.\n\nStructural priors: iSIDG employs **multiple handcrafted structural regularizers** (e.g., sparsity, connectiveness) that require careful design and hyperparameter tuning. In contrast, PreBoot-SI introduces a **unified, learnable structural prior** that is improved via iteration, offering greater flexibility, convenience, and robustness.\n\nContribution type: iSIDG contributes a **new strong baseline model**. PreBoot-SI instead provides a **general-purpose enhancement framework** that can be plugged into various VAE-based structural inference methods (e.g., NRI, MPM), improving their accuracy without modifying their core architectures.\n\nTo our knowledge, we are the first to introduce learnable priors and multi-level guidance within the structural inference domain. We respectfully ask the reviewer to reconsider the novelty of PreBoot-SI, in light of its unified, scalable design and the innovative concepts it contributes to structural inference research. A summary of the key differences between iSIDG and PreBoot-SI is provided below:\n\n| Aspect              | iSIDG [1]                                                                 | PreBoot-SI (Ours)                                                                 |\n|---------------------|---------------------------------------------------------------------------|------------------------------------------------------------------------------------|\n| Model architecture  | Iteratively prunes encoder graph based on current predictions             | Iteratively refines a separate prior network that guides both embedding and structural inference |\n| Structural priors   | Uses handcrafted regularization losses with manual tuning                 | Uses learnable structural prior and updates it via bootstrapped iteration         |\n| Contribution type   | Introduces a new standalone structural inference model                    | Offers a general-purpose framework for enhancing various existing models          |\n\n[1] Iterative Structural Inference of Directed Graphs. In NeurIPS, 2022.\n\n> **W2**: Narrow evaluation scope: Results focus on synthetic physics datasets (e.g., Kuramoto oscillators), with no validation on real-world biological or multi-agent systems. This limits claims about practical utility.\n\nWe fully agree on the importance of validating PreBoot-SI on more realistic systems. However, fully observed trajectories with reliable ground-truth interaction graphs are scarce in real-world structural inference tasks, making systematic evaluation difficult. This limitation is common in this area.\n\nTo better assess generalization, we conducted additional experiments on the DoSI benchmark [1], which simulates dynamics over empirically-derived graphs from real-world domains. Specifically, we tested on three biologically inspired datasets (15 nodes): Brain Networks (BN), Gene Regulatory Networks (GRN) and Vascular Networks (VN). Besides the Spring simulator, we also used the NetSims simulator, which models brain activity using nodes for brain regions and edges for interactions. While the trajectories are still simulated, the interaction graphs and simulator are sourced from real-world observations, thus providing a more realistic evaluation setting.\n\nAs shown in the table below, PreBoot-SI continues to yield consistent improvements over the NRI and MPM baselines and achieves **state-of-the-art performance among VAE-based models**  (under the AUROC performance metric used in DoSI). We will continue to extend the experiments on DoSI benchmark soon.\n\n| Methods                    | Springs  BN | Springs GRN   | Springs VN    | NetSims  BN | NetSims GRN   | NetSims VN    |\n|----------------------------|--------------------------|-------|-------|---------------------------|-------|-------|\n| NRI                        | 99.75                   | 90.55 | 92.68 | 99.79                    | 78.08 | 89.13 |\n| ACD                        | 99.75                   | 91.10 | 94.34 | 99.87                    | 80.18 | 80.32 |\n| MPM                        | **99.98**               | 94.02 | 96.56 | 99.95                | 76.06 | 91.18 |\n| iSIDG                      | 99.97                   | 92.91 | 96.59 | 99.91                    | 71.11 | 91.20 |\n| RCSI                       | 99.81                   | 93.01 | 97.03 | 99.72                    | 77.45 | 91.53 |\n| PreBoot-SI (NRI-based)     | 99.75                   | 91.19 | 93.77 | 99.80                    | 84.40 | 93.75 |\n| PreBoot-SI (MPM-based)     | **99.98**               | **98.95** | **99.39** | **99.97**                | **95.32** | **96.75** |\n\n[1] Benchmarking Structural Inference Methods for Interacting Dynamical Systems with Synthetic Data. In NeurIPS Datasets and Benchmarks Track, 2024.\n\n> **W3**: The scalability is concerned: The framework’s computational overhead (e.g., iterative training) and quadratic complexity (due to fully connected GNNs) hinder applicability to large-scale systems, as acknowledged in Section 6. For real-world large-scale dynamical systems, the scalability of the proposed approach is of concern.\n\nWe recognize that scalability is a key concern for structural inference in large dynamical systems, which is a common limitation across many existing models such as NRI [1] and MPM [2]. The present study aims to propose a general-purpose performance enhancement framework, and thus we evaluate PreBoot-SI under the same settings as prior baselines to ensure fair comparison, specifically small systems with fixed topologies. In fact, the primary scalability bottleneck lies in the fully connected GNN modules employed by NRI and MPM, not in the PreBoot-SI framework itself. \n\nAs noted in Section~6, PreBoot-SI is compatible with sparse or dynamic encoders. We plan to incorporate sparsity-aware structural priors to guide GNN edge construction, which has the potential to reduce inference complexity from $\\mathcal{O}(N^2)$ to $\\mathcal{O}(N)$. This direction holds promise for extending our method to large-scale or more structured systems, and we intend to pursue it in follow-up work.\n\n[1] Neural Relational Inference for Interacting Systems, In ICML, 2018.\n\n[2] Neural Relational Inference with Efficient Message Passing Mechanisms, In AAAI, 2021."}, {"review_id": "oXPRrrNtUL", "reviewer": "Reviewer_6Y2C", "summary": "This paper presents PreBoot-SI, which is a framework that designs for structural inference inspired by Neural Relational Inference. The authors propose an empirical Bayes type of approach to iteratively estimate the latent/underlying structure. In the experimental study, the authors present the performance on three synthetic physical systems: spring, charged particle, and Kuramoto oscillator. PreBoot-SI demonstrates stronger performance than baseline methods.", "questions": "1. What does the term Bootstrap mean here? Bootstrap typically refers to the procedure to resample/subsample the data and repeat the inference process to get uncertainty estimates (for example). The reviewer didn't see where Bootstrap appears in this paper. This is a huge concern from the reviewer. The reviewer believes this work is closer to EM or empirical Bayes type of approach (but not Bootstrapping). \n\n2. May the authors shorten the paragraph to describe NRI and move the model design forward?", "limitations": "Yes.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths: \n1. This topic is clearly significant for the structural inference problems. \n2. The proposed method with a learned prior (empirical Bayes style) is indeed likely to improve performances. \n\nWeaknesses: \n1. The experimental study is not extensive to support the claim. The reviewer would love to see the result working on a larger and more complex dataset. \n2. The novelty can be very limited.", "was_revised": true, "final_justification": "With a round of revision just on the definition of Bootstrap and its usage could make the clarity of this paper much better. This work is useful for this community with clear improvement and motivation and implementation.", "rebuttal": "> **W1**: The experimental study is not extensive to support the claim. The reviewer would love to see the result working on a larger and more complex dataset.\n\nIn this work, our primary goal is to introduce a general-purpose performance enhancement framework for structural inference, rather than to target specific scalability improvements. To ensure fair and direct comparison with prior methods, we adopted standard benchmark settings from previous work ([1], [2]). These settings include datasets from Springs, Charged, and Kuramoto simulators, each comprising 5 or 10 nodes.\n\nMeanwhile, we agree that broader evaluation is important. We conducted additional experiments on the DoSI benchmark [3], which simulates dynamics over empirically-derived graphs from real-world domains. Specifically, we tested on three biologically inspired datasets (15 nodes): Brain Networks (BN), Gene Regulatory Networks (GRN) and Vascular Networks (VN). Besides the Spring simulator, we also used the NetSims simulator, which models brain activity using nodes for brain regions and edges for interactions. While the trajectories are still simulated, the interaction graphs and simulator are sourced from real-world observations, thus providing a more realistic evaluation setting.\n\nAs shown in the table below, PreBoot-SI continues to yield consistent improvements over the NRI and MPM baselines and achieves **state-of-the-art performance among VAE-based models** (under the AUROC performance metric used in DoSI). We will continue to extend the experiments on the DoSI benchmark soon.\n\n| Methods                    | Springs  BN | Springs GRN   | Springs VN    | NetSims  BN | NetSims GRN   | NetSims VN    |\n|----------------------------|--------------------------|-------|-------|---------------------------|-------|-------|\n| NRI                        | 99.75                   | 90.55 | 92.68 | 99.79                    | 78.08 | 89.13 |\n| ACD                        | 99.75                   | 91.10 | 94.34 | 99.87                    | 80.18 | 80.32 |\n| MPM                        | **99.98**               | 94.02 | 96.56 | 99.95                | 76.06 | 91.18 |\n| iSIDG                      | 99.97                   | 92.91 | 96.59 | 99.91                    | 71.11 | 91.20 |\n| RCSI                       | 99.81                   | 93.01 | 97.03 | 99.72                    | 77.45 | 91.53 |\n| PreBoot-SI (NRI-based)     | 99.75                   | 91.19 | 93.77 | 99.80                    | 84.40 | 93.75 |\n| PreBoot-SI (MPM-based)     | **99.98**               | **98.95** | **99.39** | **99.97**                | **95.32** | **96.75** |\n\n[1] Neural Relational Inference for Interacting Systems, In ICML, 2018.\n\n[2] Neural Relational Inference with Efficient Message Passing Mechanisms, In AAAI, 2021.\n\n[3] Benchmarking Structural Inference Methods for Interacting Dynamical Systems with Synthetic Data. In NeurIPS Datasets and Benchmarks Track, 2024.\n\n> **W2**: The novelty can be very limited.\n\nWe appreciate that NRI and its variants have been employed in existing structural inference tasks. However, prior work faces two key challenges: (1) **difficulty in long-chain joint training** under the VAE framework, and (2) **limitations in generalization and practicality** due to the reliance on manually designed structural priors.\n\nOur method introduces a plug-in framework that enhances a range of VAE-based structural models. It addresses these two challenges by leveraging **learnable priors** to offer **multi-level guidance** throughout long-chain joint training. To our knowledge, we are the first to introduce learnable priors and multi-level guidance within the structural inference domain. Our method offers a big step toward integrating structural priors and trajectories into structural inference. We respectfully ask the reviewer to reconsider the novelty of PreBoot-SI, in light of its unified, scalable design and the innovative concepts it contributes to structural inference research.\n\n> **Q1**: The meaning of the term \"Bootstrap\". \n\nWe appreciate this important observation. The term \"bootstrapping\" indeed has a well-established meaning in classical statistics, typically referring to data resampling methods [2] for uncertainty estimation. In our work, the term “bootstrapping” aligns more closely with its general meaning—leveraging its own outputs to iteratively improve itself. Specifically, the prediction results from the previous step of SIjoint will be used in the next step for supervised training of SIprior, while SIprior will in turn guide the subsequent training of SIjoint. Therefore, we use the term \"bootstrapping\" to describe this iterative process.\n\n We acknowledge that this term may cause confusion due to its different meanings in various fields such as statistics, machine learning, and compilation techniques. Therefore, we will instead use the terms \"iteration\" and \"iterative\", which more neutrally describe the self-improvement process in our method.\n\n[2] Bootstrap Methods: Another Look at the Jackknife. Breakthroughs in statistics: Methodology and distribution, 1992.\n\n> **Q2**: May the authors shorten the paragraph to describe NRI and move the model design forward?\n\nThank you for the suggestion. We will streamline the NRI description in Section 3 and expand the PreBoot-SI design explanation to enhance clarity."}, {"review_id": "lzTm0Wz5Yq", "reviewer": "Reviewer_vjqL", "summary": "This paper points out that fixed priors of the interaction graphs limit the accuracy of neural relational inference. It proposes Pretraining & Bootstrapped Structural Inference (PreBoot-SI), a bootstrap iteration framework for structural inference. PreBoot-SI estimates the prior by training an auxiliary network on manually constructed meta-datasets with similar dynamics and known interactions. The prior module and the inference module are jointly trained to promote the structural inference accuracy. Experiments on classical synthetic datasets validate the effectiveness of PreBoot-SI.", "questions": "Q1. Constructing the meta-datasets requires knowing essential knowledge about the data generation process, which is a strong prerequisite for PreBoot-SI compared with existing methods that only assume minimal prior knowledge about the edge distribution. Can you discuss the applicability of PreBoot-SI to real-world systems when the data generation process is less understood, or can you figure out some real-world scenarios that match the assumptions of PreBoot-SI?\n\nQ2. How will the configuration of the meta-datasets affect the performance of PreBoot-SI? For example, the choice of hyperparameter lists of p and alpha, the discrepancy between the meta-datasets and the true data, etc.\n\nQ3. Please add more recent methods for comparison. See W1 for details.\n\n**Other Comments.**\n\nC1. This paper is essentially learning a structural prior network to infer pseudo labels for the structure given the trajectories. I wonder if the term \"structural prior network\" is more accurate than the somewhat vague term \"structural priors\" that seems irrelevant to the observational state series.\n\nC2. The caption of Figure 3 states that there are three experiments, but only two labeled subfigures are presented. Specifically, a small figure on the SI_joint accuracy against bootstrap round is embedded in Figure 3(b). Please consider better organizing Figure 3.", "limitations": "Constructing the meta-datasets requires knowing essential knowledge about the data generation process, which may not always be available. Hence, it is unclear if such a method is always feasible for real-world systems where the data generation process is less understood.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nS1. The motivation is clear. The writing is fluent. The formulas are well-organized.\n\nS2. Training a structural prior network on meta-datasets is novel for structural inference.\n\nS3. The experimental results on three synthetic datasets are significant. The ablation studies verify the necessity of the proposed components.\n\n**Weaknesses**\n\nW1. The comparison methods are not up to date. Please consider adding more recent methods for comparison. Here are some relevant papers with no conflicts of interest.\n\nReferences:\n\n[1] Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms. In ICML, 2025.\n\n[2] Structural Inference of Dynamical Systems with Conjoined State Space Models. In NeurIPS, 2024.\n\n\nW2. PreBoot-SI is only tested on three classical synthetic datasets. Though it is sufficient for proof of concept, testing PreNoot-SI on real-world datasets or more diverse benchmarks [3] will make the results more convincing.\n\nReferences:\n\n[3] Benchmarking Structural Inference Methods for Interacting Dynamical Systems with Synthetic Data. In NeurIPS Datasets and Benchmarks Track, 2024.", "was_revised": true, "final_justification": "The authors have addressed all my concerns. The proposed framework PreBoot-SI can generally enhance the performance of structural inference baselines, and has the potential to solve real-world problems.", "rebuttal": ">**W1**: The comparison methods are not up to date. Please consider adding more recent methods for comparison. Here are some relevant papers with no conflicts of interest.\n\nWe appreciate the reviewer’s suggestion and have carefully reviewed the suggested papers. For SGSI [1], we found that it focuses on settings with partially known structures (e.g. edges that are known to exist/not exist), which differs from our fully latent setup and introduces challenges for a fair comparison. \n\nAs for SICSM [2], the model focuses on situations of irregular sampled trajectory and incomplete observed systems, and could be compared under standard settings (complete observed system and regular sampled trajectory) as a baseline. However, to our knowledge, the code has not been released yet, so a complete empirical comparison cannot be conducted at present. As a temporary measure, we estimated AUROC values from the plotted results for Vascular Networks (15 nodes) in the figures of the paper since the paper does not report specific numerical values, and the results are shown in the table below. We will contact the paper authors for a more comprehensive comparison soon.\n\n|             | SICSM | NRI   | MPM   | PreBoot-SI (NRI-based) | PreBoot-SI (MPM-based) |\n|-------------|--------|--------|--------|--------------------------|--------------------------|\n| Springs     | ~97.7   | 92.68  | 96.56  | 93.77                   | **99.39**                |\n| NetSims     | ~95.5   | 89.13  | 91.18  | 93.75                   | **96.75**                |\n\nMeanwhile, we supplemented our experiments with results from the recent DoSI benchmark [3], which includes newer methods such as RCSI—a novel method using reservoir computing [4]. We kindly ask for the reviewer’s understanding of the practical difficulties in including every new method. Additionally, PreBoot-SI is a general-purpose enhancement framework, and can potentially be integrated with newer models in future work.\n\n[1] Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms. In ICML, 2025.\n\n[2] Structural Inference of Dynamical Systems with Conjoined State Space Models. In NeurIPS, 2024.\n\n[3] Benchmarking Structural Inference Methods for Interacting Dynamical Systems with Synthetic Data. In NeurIPS Datasets and Benchmarks Track, 2024.\n\n[4] Effective and Efficient Structural Inference with Reservoir Computing. In ICML, 2023.\n\n>**W2**: PreBoot-SI is only tested on three classical synthetic datasets. Though it is sufficient for proof of concept, testing PreBoot-SI on real-world datasets or more diverse benchmarks [3] will make the results more convincing.\n\nWe agree that broader validation is important. To further evaluate generalization, we have conducted additional experiments on the DoSI benchmark [3], which contains empirical networks derived from real-world systems. Due to time and computational constraints, we report results on the Brain Networks (BN), Gene Regulatory Networks (GRN) and Vascular Networks (VN) datasets (15 nodes). The results show that PreBoot-SI consistently outperforms the NRI and MPM baselines and achieves SOTA performance (AUROC values) among VAE-based models in the benchmark. We will continue expanding these evaluations and provide additional results on DoSI.\n\n| Methods                    | Springs  BN | Springs GRN   | Springs VN    | NetSims  BN | NetSims GRN   | NetSims VN    |\n|----------------------------|--------------------------|-------|-------|---------------------------|-------|-------|\n| NRI                        | 99.75                   | 90.55 | 92.68 | 99.79                    | 78.08 | 89.13 |\n| ACD                        | 99.75                   | 91.10 | 94.34 | 99.87                    | 80.18 | 80.32 |\n| MPM                        | **99.98**               | 94.02 | 96.56 | 99.95                | 76.06 | 91.18 |\n| iSIDG                      | 99.97                   | 92.91 | 96.59 | 99.91                    | 71.11 | 91.20 |\n| RCSI                       | 99.81                   | 93.01 | 97.03 | 99.72                    | 77.45 | 91.53 |\n| PreBoot-SI (NRI-based)     | 99.75                   | 91.19 | 93.77 | 99.80                    | 84.40 | 93.75 |\n| PreBoot-SI (MPM-based)     | **99.98**               | **98.95** | **99.39** | **99.97**                | **95.32** | **96.75** |\n\n[3] Benchmarking Structural Inference Methods for Interacting Dynamical Systems with Synthetic Data. In NeurIPS Datasets and Benchmarks Track, 2024.\n\n> **Q1**: The construction of the meta-datasets and corresponding real-world scenarios.\n\nOur framework supports two complementary pseudo-labeling strategies: (a) using synthetic meta-datasets when domain knowledge exists, and (b) using pseudo-labels directly from a baseline model (e.g., NRI) when such knowledge is unavailable. Table 1 shows that both strategies improve baseline performance. For real-world scenarios, we believe there are domains where theoretical dynamics are reasonably understood—for example, brain systems modeled via Wilson-Cowan [5], FitzHugh–Nagumo [6], or Jansen-Rit dynamics [7] in computational neuroscience. While such models may not precisely reflect all real data, they can be used to generate meta-datasets that capture domain-level trajectory–structure regularities. More broadly, this opens the door to constructing a general-purpose prior network trained across diverse simulated systems, which is an exciting direction we hope our work will inspire.\n\n[5] Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical journal, 1972.\n\n[6] Impulses and physiological states in theoretical models of nerve membrane. Biophysical journal, 1961.\n\n[7] Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns. Biological cybernetics, 1995.\n\n> **Q2**: How will the configuration of the meta-datasets affect the performance of PreBoot-SI? For example, the choice of hyperparameter lists of p and alpha, the discrepancy between the meta-datasets and the true data, etc.\n\nOur principle in constructing meta-datasets is to simulate a family of systems that share similar physical principles with the target system, but exclude the specific target instance. We selected ranges of α and p accordingly to reflect diverse dynamics. Although we have not yet conducted an exhaustive analysis of hyperparameter sensitivity (due to the combinatorial size), Figure 3(b) demonstrates that PreBoot-SI remains robust even when pseudo-labels are of very low quality—i.e., close to random. This indicates that the framework can self-correct and remain effective even if the meta-dataset is imperfectly configured.\n\n> **C1**: Is the term \"structural prior network\" more accurate?\n\nWe thank the reviewer's valuable suggestion. The SIprior we trained is indeed a structural prior network that encodes structural prior information and can provide multi-level guidance for the training of SIjoint. We will supplement this fact in the paper to emphasize the essence of our method.\n\n> **C2**: Organization of Figure 3.\n\nThank you for catching this. We initially felt that the small figure in Figure 3(b) was too sparse as a separate subfigure, so we changed it to an embedded small figure. We will supplement the experiments in Figure 3, and reorganize them into three subfigures.\n\n> **Paper Formatting Concerns**: Please cite papers in their officially published version when available, e.g., Refs [3][12][20].\n\nThank you for pointing out the formatting issue regarding the citations. We have carefully reviewed the references to cite their officially published versions [1,2].\n\n[1] Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In ACL, 2013.\n\n[2] The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. In ICLR, 2017."}, {"review_id": "Cq6IXo7FBk", "reviewer": "Reviewer_UCFY", "summary": "The problem addressed is that of full identification of the hidden interaction graph (or the adjacency matrix) of a dynamical system, which can be theoretically intractable or experimentally impractical. The authors propose a framework for structural inference in systems of interacting agents, by providing more supervision signal to the model during training. Specifically, the authors provide inputs for the trajectory embedding and encoding modules typically found in VAE-based structural inference models. Results on physical systems, along with ablation studies, are presented and discussed.", "questions": "My concerns and questions mainly concern the methodology. I am happy to increase my score if most of them are satisfactorily addressed:\n1) L151: How do you show that the model escapes from local optima?\n2) L147: Failure modes haven't been investigated. Does providing wrong structural labels (bad meta-datasets or terrible pseudo-labels) significantly hurt performance? \n3) L155: What do you mean by \"it possesses universality\"?\n4) L159: Why do SI_prior and SI_joint adopt the same GNN-based encoder architecture? Is there anything discouraging users from changing the architectures?\n5) L202: The supervision from synthetic meta-datasets appears impractical on real-world problems, but very interesting. Have steps been taken to guarantee that no simulated system is present in the test set?\n6) L220: In the bootstrapping strategy, updates of $SI_{prior}$ and $SI_{joint}$ are **alternating**. Is there any advantage the alternating minimization strategy offers over ordinary **joint** minimization, as done in [1] ?\n7) Figure 3a highlights interesting training regimes. But it should be repeated over several trials for definitive conclusions to be drawn!\n\n\n\n### References:\n- [1] Nzoyem et al., Neural Context Flows for Meta-Learning of Dynamical Systems, ICLR 2025.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Quality\nThe proposed approach and the problem addressed are very interesting. I find the bootstrapping idea particularly interesting. The improved performance over the baselines is noticeably compelling, which is expected given the added prior information.\n\nThat said, the practicality of the preBoost-SI method remains unconvincing. It requires more supervision, thereby taking away one of the main advantages of conventional NRI. For instance, we require a baseline model or meta-dataset for supervision of $SI_{prior}$, which is not guaranteed to be accessible for arbitrary use cases.\n\n## Clarity\nI find that the clarity of this paper is lacking in many ways.\n\n- L116. You mention joint unsupervised training of encoder and decoder. But as I understand, only the encoder is unsupervised, while the decoder has access to ground-truth trajectories for future state prediction, which would make it self-supervised. Could you clarify this?\n- Eq 18, do the learnable prior $q^{prior}$ and $q$ share the same weights $\\phi$ ? I'm assuming that's not the case, but it should be clarified. Similarly, I assume that the GRU used in Eq 12 is different from the one used in Eq 6. That is not clear.\n- L173 conflicts with L159. In L159, it should be clearer that $SI_{joint}$ is not simply an encoder. Essentially, only its encoder part coincides with $SI_{pior}$.  \n- The notations in the supplementary material severely diverge from those in the main text, making it rather impossible to follow.\n\nMinor concerns (no effect on rating):\n- Eq (6-9). I'm assuming the superscripts 1 and 2 indicate the hidden layer?\n- Eq (10,12). \"prior\" in italic\n- No clear Conclusion in the manuscript\n\n## Significance\nThis work is significant for the community, as it shows how beneficial adding physical priors can be. It provides a good review of NRI, and researchers and practitioners stand to benefit from it.\n\n## Originality\nI am not very familiar with the \"structural inference\" approach to learning dynamical systems, but to the best of my knowledge, the work presented here is original and differs significantly from that provided in the presented literature.", "was_revised": true, "final_justification": "The authors have resolved my issues regarding the Clarity of their work. They've removed some claims that were unsubstantiated, and performed additional experiments for other claims. The work is much stronger in terms of Clarity than the original submission.", "rebuttal": "We sincerely thank the reviewer for the constructive and detailed feedback. In the following, we address each concern point-by-point and will revise the manuscript accordingly to improve clarity and rigor.\n\n> **Quality**: PreBoot-SI requires more supervision, thereby taking away one of the main advantages of conventional NRI. \n\nWe thank the reviewer for this thoughtful observation. We would like to clarify that, similar to NRI [1], we also target situations where the ground-truth structure is unknown. To address this, we propose two complementary methods to obtain pseudo labels instead of directly using ground-truth structure labels:\n\n(a) With prior knowledge of dynamics: We simulate a synthetic meta-dataset comprising systems with similar dynamics (e.g., radial force interactions) but varying parameters. These synthetic systems are generated with structure labels.\n\n(b) Without prior knowledge of dynamics: We use a baseline model (e.g., NRI) to generate initial pseudo-labels, which may be inaccurate but still serve as informative priors. Training the baseline model also does not require ground-truth structure labels.\n\nIn both cases, PreBoot-SI progressively refines the structural estimates over iterations.\n\n[1] Neural Relational Inference for Interacting Systems. In ICML, 2018.  \n\n>**C1**: L116. 'Unsupervised' vs. 'Self-supervised' training.\n\nThe term 'unsupervised' as used in our paper follows a domain-specific convention in the structural inference literature (e.g., NRI [1]), referring to settings where no ground-truth structure labels are available. While the decoder is trained using future trajectory segments—technically making it self-supervised in broader ML contexts—this setup is commonly regarded as “unsupervised” in this field, as the goal is to recover structure from observable trajectories.\n\n[1] Neural Relational Inference for Interacting Systems. In ICML, 2018.  \n\n> **C2**: Eqs. (6, 12, 18): Clarification of weight sharing.\n\nThe learnable prior modules in Eq.18, and the GRUs in Eq.6 and Eq.12, do not share weights. These modules belong to different components—SIprior and SIjoint—and serve distinct purposes. Therefore, they are separately parameterized and trained.\n\n> **C3**: L159, L173: Clarifying architecture components. \n\nWe thank the reviewer for pointing this out. We will clarify that SIjoint includes both an encoder and decoder (typical of VAE-based models), while SIprior includes only an encoder.\n\n> **C4**: Notation inconsistency between the main text and the supplement.\n\nWe sincerely apologize for the inconsistencies in notation. We will conduct a thorough review and revise both the main text and the supplementary material to ensure notational consistency and improve readability.\n\n> **C5-7**: Minor corrections.\n\nAs the reviewer suggested:\n- We will add explanation about the superscripts in Eqs. (6–9) indeed indicate different hidden layers.  \n- We will correct the formatting to italicize “prior” where appropriate in Eqs. (10, 12).  \n- We acknowledge the omission of a Conclusion section and will include one in the revised manuscript.  \n\nWe greatly appreciate these helpful comments and believe they will significantly improve the clarity of the paper.\n\n> **Q1**: L151: How to show that the model escapes from local optima?\n\nIn VAE-based structural inference models, the encoder learns a latent structure distribution from trajectories, which is used by the decoder (e.g., GNN) to predict future states. The global optima we expect is that the encoder can predict the accurate structure, while the decoder can learn the dynamic mode of the system, thus predict accurate future trajectory based on the accurate structure.\n\nDue to limited supervision—only future trajectory loss—the encoder and decoder may collude to produce plausible predictions even with incorrect latent structures, representing a local optimum. Therefore, the accuracy of structural prediction reflects the distance from the global optima. Since PreBoot-SI significantly boosts structural accuracy, this indicates that the encoder is guided towards better latent representations and closer to the global optima.\n\n> **Q2**: L147: Are there failure modes when the pseudo-labels are poor?\n\nFigure 3(b) demonstrates the robustness of PreBoot-SI to poor pseudo-labels. Even when given with randomly generated labels (50\\% accuracy), the model still converges to the NRI baseline performance, indicating that poor initial labels do not significantly degrade final outcomes.\n\n> **Q3**: L155: The meaning of “universality”?\n\nBy “universality”, we refer to the general compatibility of our approach with VAE-based structural inference models. Most such models share common design patterns—trajectory embedding and ELBO loss with KL divergence—and PreBoot-SI integrates with them via two minimal modifications: feature-level guidance (embedding concatenation) and prior replacement (KL divergence term). Thus, our framework can be adapted to many existing models by swapping the backbone architecture of SIprior and SIjoint.\n\n> **Q4**: L159: Why do SIprior and SIjoint adopt the same GNN-based encoder architecture? Is there anything discouraging users from changing the architectures?\n\nThe encoder architectures of SIprior and SIjoint are designed to be the same because their embeddings are concatenated during training. Using a consistent architecture ensures that SIjoint can interpret the structure-relevant features extracted by SIprior. Nonetheless, the architecture itself is flexible, and both modules can be simultaneously replaced—for instance, we implemented PreBoot-SI variants based on both NRI and MPM. This compatibility is a core strength of our framework.\n\nTo test this, we implemented a \"hybrid architecture\" version of PreBoot-SI that uses RNN encoder in MPM to build SIprior and GNN encoder in NRI to build SIjoint, and validated it on Gene Regulatory Network dataset (15 nodes, NetSims simulator) in the DoSI benchmark. The results in the table below show that even build SIprior with more powerful encoders, the performance (AUROC value) of PreBoot using a hybrid architecture is still lower than the standard NRI-based PreBoot-SI, reflecting the necessity of architecture consistency.\n\n| Models                            | NRI    |MPM| PreBoot-SI (NRI-based) | PreBoot-SI (MPM-based) | PreBoot-SI (hybrid architecture)|\n|----------------------------------|--------|--|-------------|----------------------------------|--------------------------|\n| AUROC values (in %)              | 78.08  |76.06| 84.40       | 95.32                            | 82.14|\n\n> **Q5**: L202: Does meta-dataset supervision introduce test leakage?\n\nWe ensure that no target systems from the test set are included in the meta-dataset by strictly excluding the target system corresponding to the test set when generating the synthetic meta-dataset.\nAs for real-world scenarios, we envision using theoretical models to synthesize meta-datasets. In neuroscience, for instance, several dynamical models like Wilson–Cowan [1], FitzHugh–Nagumo [2], and Jansen–Rit [3] offer plausible ways to simulate biologically meaningful dynamics and construct domain-inspired meta-datasets, even if the exact structure and mechanism are unknown.\n\n[1] Excitatory and Inhibitory Interactions in Localized Populations of Model Neurons. Biophysical Journal, 1972.  \n\n[2] Impulses and Physiological States in Theoretical Models of Nerve Membrane. Biophysical Journal, 1961. \n\n[3] Electroencephalogram and Visual Evoked Potential Generation in a Mathematical Model of Coupled Cortical Columns, Biological Cybernetics. 1995.\n\n> **Q6**: L220: Why use alternating updates instead of joint minimization as in [1]?\n\nWe appreciate the reference to Nzoyem et al. [1]. In our framework, we adopt full alternating training between SIprior and SIjoint instead of performing joint updates within the same epoch. Our rationale is threefold:  \n\n(1) Design motivation: We hypothesize that more accurate structure priors from SIprior help improve SIjoint, and vice versa—hence alternating full updates facilitates mutual refinement and reflects the above motivation more intuitively;  \n\n(2) Stability: Since SIjoint receives input from SIprior in the embedding layer, we hope that such input is stable, as it is generally better to provide a fixed distribution of input for the neural network to learn specific patterns;\n\n(3) Optimization: Alternating updates may help escape the local optima described above.  \n\nTo validate this, we implemented a variant following the strategy in [1], where both modules are updated in each epoch. We ensured fairness by adjusting the number of epochs and learning rate schedules. Experiments on the 15-node Gene Regulatory Network in DoSI show that while this variant still outperforms NRI, it lags behind the original PreBoot-SI. We will include this result in the supplementary material.\n\n| Models                            | NRI    | PreBoot-SI (NRI-based) | PreBoot-SI (NRI-based, joint minimization) |\n|----------------------------------|--------|-------------|----------------------------------|\n| AUROC values (in %)              | 78.08  | 84.40       | 81.89                            |\n\n[1] Neural Context Flows for Meta-Learning of Dynamical Systems. In ICLR, 2025.\n\n> **Q7**: Figure 3a highlights interesting training regimes. But it should be repeated over several trials for definitive conclusions to be drawn!\n\nWe appreciate the suggestion and will repeat the visualization in Figure 3(a) (along with those in Figure 3(b)) over multiple datasets to draw more definitive conclusions. We agree that the observed training dynamics—initial imitation of $\\text{SI}_{\\text{prior}}$ followed by refinement—are interesting and deserve further analysis.\n\nOnce again, we thank the reviewer for the thoughtful and constructive suggestions. We believe that the proposed revisions and clarifications will improve the clarity and readability of our work."}]

Author rebuttal:

Meta-review: Five knowledgeable reviewers recommend Accept, and I recommend Accept. The authors provided new experimental evaluations for their method during the rebuttal and clarified several claims, which have improved the quality of the work. I ask the authors to include these additional experiments and discussion in the Camera Ready version.
Decision: Accept (poster)