Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Authors: Aoran Wang
Xinnan Dai
Jun Pang
Venue: ICML 2025 poster (2025)
Keywords: Structural Inference
Prior Knowledge
Dynamical Systems
Primary area: applications->everything_else

Abstract: Existing methods for inferring latent relational structures struggle to integrate partial prior knowledge, such as known edges, node-degree constraints, and global sparsity, without destabilizing training or conflicting with probabilistic assumptions. We propose Soft-Gated Structural Inference (SGSI), a VAE framework that seamlessly incorporates domain constraints via (1) soft gating with learnable edge masks to preserve gradients, (2) cloning-clamping of deterministic edges to avoid distributional conflicts, and (3) adaptive regularization to balance data-driven learning with domain constraints. By excluding known edges from stochastic inference, SGSI reallocates capacity to uncertain interactions, optimizing the information bottleneck trade-off. Experiments on 16 datasets show SGSI improves edge recovery by up to $9$\% AUROC over baselines, scales to larger graphs ($94.2$\% AUROC), and maintains stable training. SGSI bridges domain expertise with data-driven learning, enabling interpretable and robust structural discovery in dynamical systems.

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Aoran Wang'
Xinnan Dai Jun Pang 13
Abstract
ing and inference of these structures, they often overlook key practical reality: domain experts frequently possess partial prior knowledge on the structure, such as known interactions in biochemical networks (Pratapa et al., 2020) or kinematic constraints in multi-agent systems (Brasó & Leal-Taixé, 2020). These kinds of prior knowledge are ig- nored, wasting opportunities to guide inference and forcing models to "re-discover" known edges, wasting capacity on irrelevant uncertainty. However, integrating such knowl- edge into structural inference frameworks is challenging, as naively overwriting edges or enforcing rigid constraints risks destabilizing training, contradicting the probabilistic assumptions of VAEs, and conflicting with the information- theoretic principles underlying these models.
Existing methods for inferring latent relational structures struggle to integrate partial prior knowl- edge, such as known edges, node-degree con- straints, and global sparsity, without destabilizing training or conflicting with probabilistic assump- tions. We propose Soft-Gated Structural Inference (SGST), a VAE framework that seamlessly incor- porates domain constraints via (1) soft gating with learnable edge masks to preserve gradients, (2) Mooing--lamminn of deterministic edges avoid distributional conflicts, and (3) adaptive regular- ization to balance data-driven learning with do- main constraints. By excluding known edges from stochastic inference, SGSI reallocates capacity to uncertain interactions, optimizing the information bottleneck trade-off. Experiments on 16 datasets show SGSI improves edge recovery by up 9% AUROC over basclines, scales to larger graphs (94.2% AUROC), and maintains stable training. SGSI bridges domain expertise with data-driven learning, enabling interpretable and robust struc- tural discovery in dynamical systems.
Existing approaches struggle reconcile prior knowledge with data-driven learning. For instance, forcibly clamping edges to fixed values disrupts gradient flow during back- propagation, while imposing exact sparsity or node-degrce constraints creates conflicts with the variational informa- tion bottleneck (VIB) objective (Alemi et al., 2017), which balances compression of irrelevant information against pre- dictive accuracy. These issues often manifest as failed con- vergence suboptimal inference, limiting the utility of structural inference in practice.
1. Introduction
We address these challenges with Soft-Gated Structural Inference (SGSI), a framework that integrates partial prior knowledge through three principal components. First, a soft gating mechanism, where each edge is assigned a learn- able parameter a. E [0, 1]. smoothly masks interactions while preserving gradient flow. Second, known edges are cloned and clamped to deterministic values (1 for present, O for absent) during forward passes, avoiding in-place mod- ifications and excluding them from the Kullback-Leibhe (KL) divergence calculation. Third, global sparsity and node-degree constraints are enforced via adaptive penalties, allowing deviations when supported by data. By treating known edges as deterministic (zero entropy) and uncertain edges stochastic, SGSI aligns with the VIB principle: it compresses irrelevant interactions while preserving predic- tive capacity for uncertain ones, optimizing the trade-off between domain priors and data-driven discovery.
The ability to infer latent relational structures, such inter- actions between particles in physics (Kwapień & Droždž, 2012; Ha & Jeong, 2021; Wu et al., 2024), agents in robotics (Brasó & Leal-Taixé, 2020; Li et al., 2022), or genes in biological systems (Tsubaki et al., 2019; Pratapa et al., 2020), is critical for modeling complex dynamical sys- tems (Birkhoff, 1927; Katok & Hasselblatt, 1995). While recent advances in variational autoencoder (VAE)-based methods (Kipf et al., 2018; Alet et al., 2019; Chen et al., 2021; Löwe et al., 2022; Wang & Pang, 2022; Wang et al., 2023; Wang & Pang, 2024b) have enabled end-to-end learn-
'Department of Computer Science, University of Luxem- bourg "ShanghaiTech University "Institute for Advanced Stud- ies, University of Luxembourg. Correspondence to: Jun Pang <jun.pang @ @uni.lu>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Experiments across dynamical systems and biological net- works demonstrate SGSI's advantages. When partial prior

---

Guided Structural Inferenee: Leveraging Priors with Soft Gating Mechanisms
knowledge is available, SGSI improves edge recovery by up to 9% in AAUROC score compared to VAE baselines. en- ables smooth scaling to larger graphs with 94.2% AUROC, and it maintains stable training where naive implementa- tions diverge. Theoretically, SGSI's booning-nnd-lampping strategy "frees" -eoormatioonttheortiic capacity for uncertain edges, avoiding distributional conflicts that plague naive ap- proaches. Practically, inferred structures align with domain expectations, enhancing interpretability without sacrificing predictive accuracy. Our contributions are threefold:
global sparsity, and node-degree constraints without any in-place modifications or distributional conflicts.
3. Preliminaries
3.1. Notation
We consider N nodes with observed states (xi(t)ji = N; t= I....,T} over discrete time steps t E [1...., TT. and each 2;(t) E R Moreover, we com- bine all the time steps into a single trajectory for node i: X; [L), (2)...... z (T)) E RdXT. We write X E X for the collection of all node trajectories, i.e. One typical shape is X E RNTxd if we store them in a 3D) tensor. We use a latent directed graph, represented by an adjacency matrix E {0, 1 where A.j = 1 indicates a directed edge from node i to node j (i.e., +))). j). We define E N2 potential edges, enumerated by e E (I....,E). tion goals, we separate the time-series X = (xi,.... X into past observations Xpast = {x),..., Nto h. and future targets: Xfuture = (xto+h,..... xr] according to time in.
A novel approach with gating mechanisms, clamp- cloning operation and adaptive regularization that inte- grates prior knowledge (known edges, sparsity. node degrees) into VAES without gradient instability or dis- tributional conflicts.
X =
A theoretical framework linking SGSI to the VIB principle, showing how deterministic and stochastic edges optimize the compression-predde prediction trade-off.
E). Moreover, depending on the predic-
Empirical validation across physical, biological, and multi-agent systems, showcasing the performance of SGSI comparing with baseline methods.
3.2. Structural Inference with VAEs
In this work, we aim to learn latent graph structure that ex- plains the observed data X for N interacting nodes. Specifi- cally, we posit a latent adjacency Z capturing which nodes influence which others, where Z can be discrete ((0, 1) or continuous ([0, 1) Each coordinate Z. denotes the potential presence or weight of edge among E total edges. We adopt standard VAE approach (Kipf et al., 2018; Alet et al., 2019; Webb et al., 2019; Chen et al., 2021):
2. Related Work
VAE-based Structural Inference. Many works use vari- ational autoencoders to learn latent graph structures for interacting systems (Kipf et al., 2018; Alet et al., 2019;
Chen et al., 2021; Wang & Pang. 2022; Wang et al., 2023). For example, NRI (Kipf et al., 2018) applies a GNN- decoder to predict future states given a learned adjacency. Key advancements include factorizing interactions for multi- relational systems systems (Webb et al., 2019), integrating efficient peessag--passing (Chen et al., 10, incorporating modular meta- meta-learning (Alet et al., 2019), iteratively prun- ing indirect connections (Wang& & Pang, 2022), developing structural inference with reservoir computing (Wang et al., 2023), and estimating partial correlation coefficients based on node embeddings (Wang& & Pang. 2024b). Yet, they lack systematic support for known-absent edges, node-degree limits, or global sparsity, and often rely on brittle adjacency overwrites that disrupt training dynamics.
We treat Z as a random adjacency in [0, 1 or 10. A VAE comprises an encoder go (x(Z X) and a decoder Po (X future 7,X pp along with KI. term that encour- ages simpler adjacency. The encoder comes with param- eter o, and it produces parameters } for each edge e, which can be interpreted as logits in a Bernoulli or Gumbel- softmax distribution: qa 1 (Be),, or more complex variants. The decoder comes with parameter 0. Given the learned adjacency Z and the past node states X Xpast- the decoder predicts the future states X We obtain the adjacency from the latent space of the VAE. The VAE objective for structural inference typically includes part for prediction and KL term:
Knowlede Approaches. Some frameworks can handle some prior knowledge, e.g., ALaSI (Wang & Pang, 2023) integrates known-present edges via active learning, or SICSM (Wang& & Pang, 2024a) can fix certain present edges. However, they lack flexible ways to handle absent edges, node-degree constraints, or global sparsity. Others rely on naive adjacency overwrites that can break gradient flow or create contradictory KL signals.
LvvE( 0) = E paan log pa Xiuture I2, Xpast)
Prediction Loss
+ B DKL (Ku(a (1s p(Z))
(I)
KL Regularizer
where the prediction loss encourages the learned adjacency to be informative about the next-step states or about recon- structing the entire sequences, and the KI. term encourages
We address these challenges by introducing soft gating, a mechanism that systematically accommodates known edges,
2

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
X Xpst
Xinure
Encoder
Node-to-Edge
Edge-to-Noda
Node-to-Edge
Decoder
f
ft
ft
go(2
Mechanism
{
Soft Gating Mechanism
Objective Function
P
Knowm-Present Edges Known-Absent Edges
pred + DKL
Epanntty
des
Sigmoid
{are)
Uncertain Edges
Global Sparsity
Node Degree
Constraints
Figure 1. (Above) The overview of the whole pipeline. (Boftom) Details about the soft gating and objective functions in this work.
qo (Z) Xpast) to remain close to prior p(Z), typically push- ing for a simpler or more compressed adjacency. The param- eter 3 balances how strongly we emphasize compression against prediction, drawing parallels with the information bottleneck principle (Tishby et al., 1999).
Sparsity or Global Edge Constraints: In many settings, the number of active edges in a system is expected to be small (e.g., a physical system with local interactions only, or network known to mostly disjoint). Alternatively, one might only have an approximate fraction of edges p that remain active. This can be enforced via penalty the sum of gating values or more rigid constraint the total number of edges allowed.
While existing methods infer fully unknown adjacency, inte- grating partial prior knowledge poses challenges: forcibly overwriting edges disrupts gradient flow, and fixing edges to deterministic values contradicts latent uncertainty assump- tions, creating conflicting KL signals. SGSI resolves this via soft gating, which seamlessly enforces constraints while preserving gradients and distributional consistency.
Node-Degree Constraints: Certain nodes might be known "hubs", or they might have out-degree (or in-degree) capped by a known property. For example, in a system where each node can only maintain three outgoing connections due to bandwidth limits, node might have exactly two neighbors in ring-lite like structure. We might penalize or clamp the out-degree E, Ziz to match target degree deg,
3.3. Partial Knowledge and Adjacency Constraints
Real-world systems exhibit diverse edges, including known, constrained, and uncertain, yet existing methods treat all edges uniformly latent, causing conflicts. SGSI integrates partial knowledge (known present/absent edges, sparsity, node degrees) to resolve inconsistencies and enhance infer- ence accuracy. We refer to such knowledge as partial or prior adjacency constraints in the following forms:
A naive way to incorporate known edges is to forcibly over- write the VAE's Sigmoid or Gumbel-softmax outputs for (i j) E+ with 1 or for E with O. However, this often leads in-place modification confficts in modern autograd frameworks, as the same tensor is needed for gradient com- putations. Moreover, if certain edges are forcibly set to 0 or without adjusting the KL term (or removing them from the latent distribution), the prior-latent assumption that all edges are random violated. The model may receive contradic- tory signals: on one hand, the KL encourages distribution every edge; on the other hand, are forcibly clamping some edges to fixed values. To address these issues in gradient-friendly manner, our approach uses soft gating and is discussed in detail in Section 4.
Known-Present Edges: A subset E+ S {(i +))
A < N} of edges deemed definitely present. For instance, certain couplings physically guaranteed to exist (e.g., gravitational link in a solar system), or certain agent- to-agent channels are known to be always active. We want to ensure any latent adjacency variable Zig for (i g+ effectively "on" (or near in a continuous relaxation).
Known-Absent Edges: Another subset E- S ((i iSi N) is known to be absent forbidden. This may arise if domain rules forbid certain interactions (c.g., node i cannot influence node j under the laws of the system), or if data from external knowledge confirms no direct link. Thus, for (i j) E E-, the adjacency variable Zig should be near exactly zero in a discrete adjacency.
4. Structural Inference with Soft Gating
SGSI builds upon the idea that VAB-based structural infer- ence be interpreted through the lens of the VIB. In this section, detail how embed soft gating within VAE framework, incorporate partial knowledge, and reconcile it
3

---

Guided Structural Inferenee: Leveraging Priors with Soft Gating Mechanisms
with the IB principle. Fig. illustrates the overall pipeline.
add a KL or information bottleneck penalty on &. striking balance between accurate prediction and minimal adjaceney. However, the details of this are often shaped by the choice of prior p(Z). In practice, we simply have 3-weighted KL term (Eqn. 1) for the gating distribution.
4.1. Learned Gating Overview
We start with a VAE-based structural inference setup where the latent variable Z represents the adjacency among N nodes. Instead of treating each edge strietly binary, we in- troduce continuous gating parameter 0g E R per potential edge E (l...., E). Thus we obtain gating values:
4.2. Incorporating Partial Knowledge
A central contribution of this work is to allow partial domain knowledge about edges, global sparsity or node degrees to be injected into the gating approach without causing in in-place modifications or contradictory distributions. SGSI addresses this via gradient- clamping and adaptive regularization.
ae = a(fe),
where (-) is the sigmoid functiio. Each ae E [0, 1] acts as a soft mask, suppressing irrelevant edges (a. - 0) or acti- vating critical ones (a,- - 1). Besides that, the remaining part of the network is setup as the following:
Known-Presen and Known-Absent Edges. If an edge is present according to the domain knowledge or prior knowledge. i.c. e E Et, we want de ~ 1. However, directly overwriting the Sigmoid output 0e in-place can break gradient flow. Instead, we use the following steps
Encoder (Z =
where fenc.ó is implemented by GNN that acts on the fully conditioned graph. Given node observations X
X ,,..., NNJ. it outputs logits & for each po-
1. Compute O2 = (8a) for all edges.
tential edge of the graph.
2. Clone Oe into O'clamped
Node Embedding:
h = (X))
(1)
(3) 3. Clamp in detamped: for E+, set actampodle] 1.
4. Freeze &e for e E E U (e.g.. E. - too for E+).
(1) = fa ((h (1) h D.
Node-to-edge:
(4) 5. Exclude clamped edges from the KL divergence penalty,
f
Gating 1:
Cj = h ac
(1)
(5) Cloning Oe ensures gradient computations use the original
J = fo Σ e.))
(6) Oe, while aclamped enforces domain constraints during for- ward passes. Skipping KL terms for known edges avoids conflicting signals between the prior and clamped values.
Edge-to-node:
h (2) / (1) 22) I))),,
Node-to-edge:
(7)
Gating 2:
ej h/ a
(2) (2)
Global Sparsity. Beyond edge-by-edge knowledge, we allow global constraints to be inserted via adaptive penalties. Let Og or the Oslamped be the final gating values for each edge, depending on whether the edges appear in the prior set or not. We define the penalty for sparsity as:
where [ defines concatenation of two tensors. Sembed is the embedding network for input features, f!") and f!?) node-to-edge pessage-ppassing networks, and fo is the edge-to-node operation. Each edge's message is multiplied by Gc. thus "turning on" edges close to 1 and suppressing edges near O. We then model posterior over edges as
Saparsity
Xsparsity
Oke
(11)
g / x)= = softmax (2)
(9)
encouraging the sum of all gating values be small. Asparsity is the weight of the sparsity penalty. Sometimes we want exactly (or approximately) pE edges active, where E is the total number of potential edges. For example, = 0.1 means 10% edges. We achieve this by comparing the sum of gating E. Oge to pE:
where e summarizes the parameters of the neural networks in Eqn. 3-8. Similar to approaches from (Kipf et al., 2018;
Wang & Pang, 2022; Wang et al., 2023), additional random- ization such as Gumbel-Softmax is used.
Decoder Po (Xtuuure| Z, ppast) takes in node states and ad- jacency (c(?)) from Eqn. 9 to predict nodes' subsequent states, which are influenced by edges scaled by ae- Fol- lowing (Kipf et al., 2018; 8; Wang & Pang, 2022; Wang et al., 2023), we use a message-ppssingg framework:
Esparsity
Asparsity
Σ are -Pp-
(12)
This is a soft approach as the model is free to deviate if the data demands bit more or fewer edges, but it pays a penalty in proportion to how far it strays from p.E
h (t+1) = xz +t)+ fembed
Σ Ery
(z) (Lx,
Node Degree Constraints. In many real-world settings, we may know each node i has an exact out-degree k, or sim- ilarly, an in-degree constraint. For example, a node might precisely need 2 out-connections based on domain rules. We incorporate these constraints in a soft manner rather
(10)
where p and f are multilayer perceptrons (MLPs)
to embed the features. In a standard VAE setting, we also
4

---

Guided Structural Inferenee: Leveraging Priors with Soft Gating Mechanisms
than forcing exact rewrites, which could lead to in-place modifications and contradictory distributions. Following the notations in Section 3.1, interpret ae as how "active" edge is. If edge corresponds to (i + j), then node i is the sender. We have the out-degree and in-degree of i:
Choosing Hyperparameters.
KL Weight 3: Balances prediction accuracy against model complexity. We typically begin with 3 1.0 (a balanced trade-off). For very sparse or low-data scenar- ios, we may reduce 3 (e.g.. 0.1) to prioritize prediction. Conversely, in settings demanding strong regularization or interpretability, 3 can be increased (e.g.. 2.0 or 5.0).
outdeg
((1-1))-
(13)
indeg(1) = Σ ((-+)))
(14)
Sparsity Weight Asparsity: Controls adherence to global sparsity. For exact sparsity ae R pF), set Aspansity [0.1, 1.0]. For soft encouragement, use smaller values (10-3 30-3/0 to 10-
j=1
Depending on whether we have an out-degree or in-degree constraint, we focus on one of these sums or both. Suppose
we want node i to have out-degree exactly kg. We can define:
M
Ldog-
Adeg
Σ
outdeg(i) - ki
(15)
Degrec Penalty Adeg: Similarly tested in [10-4, 10- 10-2). Larger Adeg enforces node-degree constraints more strictly, while smaller values let data cues override the prior knowledge. We pick value based on small vali- dation set or best AUROC for edge recovery.
where M represents the set of nodes having out-degree con- straints and is a subset of all N nodes. Let M C (I....,N N be the set of nodes with out-degree constraints, and let Adeg be the penalty weight. This term softly enforces Σ, 03-+j x kü but allows deviations if supported by data (at a penalty cost). For in-degree constraints, replace outdeg(i) with indeg(i). If some nodes have out-degree constraints and others have in-degree constraints, handle each node independently and sum the corresponding terms.
In practice, we find a brief grid search or Bayesian opti- mization on (3, Asparsity Adeg) ) sufficient to find hyperpa- rameters that yield both high-fidelity adjacency and stable training for the experiments in this work.
4.4. Interpretation via VIB
4.3. Objective Function
SGSI formalizes the IB principle by explicitly separating known and uncertain edges in the latent adjacency Z. Let X denote observed node trajectories, and Z represent the latent graph with three disjoint subsets:
The training objective balances prediction accuracy, model complexity, and adherence to prior knowledge. The predic- tion loss is the same as that in Eqn. 1:
e+: Known-preseat edges (fixed to 1).
Eped log po (future Celampedd Xpast).
(16)
E- Known-absent edges (fixed to 0).
However, the KL term may vary depending on the prior knowledge, If certain edges are truly fixed, we remove them from the KL computation, avoiding contradictory signals. This is straightforward if we treat those edges as having zero entropy (delta function) in the distribution, or if we skip them entirely from the random portion. For partially known constraints (like node degrees), we keep the normal KL. but apply the external penalty. This synergy ensures a stable co-existence of variational inference and domain constraints. Therefore, based on the prior knowledge we have, we update the DKL term in Eqn. accordingly:
U: Uncertain edges (learned from data).
Posterior and Prior Factorization. The encoder go (Z(x) and prior p(Z) factorize as:
L 2 XX=
II 96 (Ze
II 8(2. =1 II 8(2. - 0), (19)
eEGl
3EET
CEE-
v(Z)
II eEU
pl ZZ.)
8(2e = 1)
II 8(2. = 0),
EEE+
(20)
DKL
Σ'LL KL EKL(ZOOE CEu
(Ee
x) || P(E.))
(17)
where 8(-) enforces deterministic values for known edges. This factorization ensures:
are excluded from the stochastic portion of Z, avoiding contradictory signals between data and prior knowledge.
Known edges
u (1, E3 (E+UE) represents the set of uncertain edges, & denotes the variables in latent space. We then combine the penalties for known degrees or global sparsity with the prediction loss and B-weighted KL:
Uncertain edges (l() remain random, allowing the model to learn their distributions.
L = Lpred + DKL + Lsparsiy + Loog.
(18)
KL Divergence Simplification. The KL. divergence be-
5

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Springs Simulations
NerSims
TF
BF-CV
SALHIC.
-10
50
20
30
30
20
50
Koown
Karsea
VN_SP_15
VN_SP_30
VN_SP_50
VN_NS_15
VN_NS_10
VN_NS.N
VN_NS_10
10
AALROC
10
MPM
50251
Figure 2. The average A AUROC values of ten runs of baselines and SGSI the datasets with different percentages of known present edges. The shadings represent the standard deviations.
Springs Simulations
NetSims
TF
BF-CV
10
AAURIC
10
20
20
VN. SP 15
VN SP 30
VN SP 30
VN SP 100
VN NS. 13
VN NS 30
VN NS 30
VN NS 100
SAUROC
30
10
20
30
40
30
10
20
10
20
30
40
50
10
20
30
40
50
10
20
10
20
30
40
50
10
20
30
Kassen Altennc in
Kaxrea Abocace
Ka Alsctoc 10
Kasner
Alo-2222 18.5
Kauem Absexae
Kraona Alecaue i6
Kaaona Absettc in
NRI
ACD
MPM
15EDG
RCSI
AlaSI
-
S051
Figure 3. The average AAUROC values of ten runs of baselines and SGSI on the datasets with different percentages of known absent edges. The shadings represent the standard deviations.
tween posterior and prior reduces to:
trade-off:
DKL (x Σ DKL (Z. Ip(Z)))
min
2(X
I(2;))
Uncertain edges
Prediction
(21)
since (o 0 for known edges, aligning with IB:
The soft gating 0. (00)) directly implements this IB- driven compression the known edges (ae 1 0) are deterministic, contributing nothing to T(X; Z). Besides, uncertain edges (a,, E (0, 1)) are regularized by the KL term and sparsity penalties, limiting their information content.
Compression (T(X; Z): The KI. term penalizes extrane- ous information stored in uncertain edges. Known edges consume no "bits" due to their deterministic nature. Prediction (1(Z; Y)): The prediction loss ensures Z re- tains edges critical for predicting Xuttur-
5. Experimental Results
Theoretical Implications. By delta-encoding known edges, SGSI achieves two key properties:
In this section, we evaluate the proposed soft gating ap- proach on various datasets. We aim to investigate accuracy, graph quality, robustness, flexibility and the information bottleneck perspective of SGSI.
1. No Contradictory Signals:
Known edges are excluded from the random latent space, preventing conflicts be- tween prior assumptions (e.g., uniform prior) and do- main knowledge.
5.1. General Settings
2. Freed Capacity: The model allocates information- theoretic "bits" only to uncertain edges, optimizing the
Datasets.
Our study first evaluates the SGSI model on two established structural inference datasets: the Spring
6

---

Guided Structural Inferenee: Leveraging Priors with Soft Gating Mechanisms
Simulations dataset (Kipf et al., 2018), which simulates dynamic interactions of balls connected by springs within symmetric setting, and the NetSim dataset (Smith et al., 201 1b), which consists of simulated Mloodoxxyeen-eevel. dependent imaging data from various brain regions in an asymmetric network. Both datasets include 10 nodes, with Spring Simulations offering four-dimensional features and NetSim mee-imensional features at each timestep, initially sampled at 49 regular intervals.
the area under the receiver operating characteristic (AU- ROC) curve, focusing on the accuracy of the inferred adja- cency matrix. All experiments were conducted on a single NVIDIA Ampere 40GB HBM graphics card, paired with 2 AMD Rome CPUs (32 cores @ 2.35 GHz). For more details about the experiments, please refer to Appendix E.
5.2. Varying Amount of Known Edges
To assess how partial knowledge impacts structural infer- ence, we evaluate each method under varying percentages of known-absent (10%-50%) and known-present (10%-50%) edges, and demonstrate the results in Fig. 2 and Fig. 3. We measure improvements in structural inference quality via AUUROC, representing the change in AUROC relative to the same method without any prior knowledge. Higher AAUROC indicates a stronger benefit from partial adja- cency constraints. Across all baselines that rely on naive overwrites, improvements in AAUROG typically are neg- ative. The forcibly pinned edges begin to contradict the latent distributions expected by these frameworks. Methods that forcibly zero out edges often find it easier eliminate false positíves, leading to negative AUROO gains. For known-present edges, certain baselines, especially ones that strongly rely on a random adjacency distribution such as NRI, ACD, iSIDG and RCSI, struggle to fully accommodate forced 1s if that set grows large.
Furthermore, we examined six directed synthetic biological networks (Linear, Linear Long, Cycle, Bifureating, Trifur- cating, and Bifurcating Converging) as outlined in (Pratapa et al., 2020), with abbreviations LI. LL, CY, BF, TF and BF- CV, respectively. These networks simulate developmental trajectories in differentiating cells using BoolODE (Pratapa et al., 2020), capturing me-dimensional mRNA expression levels over 49 timesteps.
We also incorporated data from the StructInfer Bench- mark (Wang et al., 2024), focusing on "Vascular Networks' (VN) with node counts ranging from 15 to 100. These datasets, named under the categories Springs (SP) and Net- Sims (NS), were selected for their complex and varying underlying graph structures, providing robust platform to validate the efficacy of the SGSI model.
Baselines. To evaluate the performance of SGSI, we com- pared it against a suite of state-of-the-ane models:
While we leave global sparsity or node-degree constraints for the next section, it is already clear that naive in-place overwrites for known edges do not scale gracefully. In con- trast, SGSI with the soft gating approach, which skips or reduces KL for clamped edges, consistently reassigns ca- pacity to the uncertain edges without harming gradient flow. This synergy yields higher AAUROC at higher knowledge fractions, which is a point we underscore once additional constraints come into play. These results confirm that most baselines (NRI, MPM, ACD, iSIDG, RCSI) can not handle any partial adjacency knowledge via overwrites and exhibit remarkable negative gains at higher known-edge fractions. ALaSI and SICSM integrate partial knowledge more di- rectly, yet remain bound by their inability to handle large sets of forced edges or forcibly absent edges (in the case of SICSM). Overall, this points the need for more system- atic, gradient-friendl method that properly excludes known edges from the random portion of the adjacency, which is a motivation leading directly to our soft gating solution.
NRI (Kipf al., 2018): a VAE-based model for unsuper- vised relational inference.
MPM (Chen et al., 2021): employs a VAE framework with a relational interaction mechanism and spatio-temporal message passing.
ACD (Löwe et al., 2022): utilizes shared dynamics to infer causal relations within datasets.
iSIDG (Wang& & Pang, 2022): iteratively refines adjacency matrices to enhance directional inference.
RCSI (Wang al., 2023): integrates reservoir computing for efficient structural inference.
ALaSI (Wang& & Pang. 2023): a structural inference method based on deep active learning.
SICSM (Wang& & Pang, 2024a): a conjoined state space model for complex input data modeling.
Since NRI, MPM, ACD, iSIDG and RCSI do not provide an official way of integrating prior knowledge, we just rewrite the adjacency matrix in their latent spaces to in- tegrate known present/absent edges. ALaSI can have known present/absent edges integrated by default, and SICSM can only have known present edges integrated. Yet we did not find a possible way of having global sparsity and node de- gree constraints to be integrated into any of these baselines.
5.3. Global Sparsity and Node Degree Constraints
In Table 1, we compare the baselines, which lack any official support for partial knowledge, to SGSI under four variants: (1) SGSI w. Sparsity: A mild penalty on the total adjacency sum against the known sparsity. (2) SGSI w. In-deg.: Each node's in in-degree softly constrained by domain knowledge.
Metrics. Similar to prior work, the comparative effective- ness of these methods was quantitatively assessed using
7

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Table 1. BBsseline AUROC results without prior knowledge compared to SGST variants: SGST w. Sparsity (global sparsity), SGSI
In-deg. (known in in-degress). SGSI w. Out-deg. (known out-degrees) and SGSI w. Both-deg, (both in- and out-degrees).
Dataset
Method
VN_SP_15
VN_SP_30
VN_SP_50
VN_SP_100
VN_NS_15
VN_NS_30
VN_NS_M
VN_NS_100
NRI
94.58+ u.0
95.12+ ou1
94.65 u.z
89.17+ + ouz
90.31+ om
74.64+ 0.4
69.78+
68,80+ oaz
MPM
96.561
39.711000 0044
85.071
84.561 0.03
91.181 0.01
83.37 0.03
72.66. 0.04
70.341 0.03
ACD
94.34+ u:0
93.73+
87.54+
90. ous
80.32+ o.12
65.36+ 0.86
69.01+ 0.03
68.
SSIDG
96.59 0.02
95.66.100 001
95.72.1 0.02
85.07 0.02
91.20 0.02
78.08.1 0.06
73.68 0.02
68.81 0.02
RCSI
97.03= n.01
95.31 001
94.48 0.02
90.72+ ouss
91.53+ o.22
82.27+004
74.08 0.02
70.29+ 0.03
ALaSI
96.27 4: 0.02
95.30 0.01
96.43 0.02
91.05 o.0s
89.03 :: 0.04
80.35 1: 0.06
75.31 3: 0.04
73.544 0.04
SICSM
97.70= 0.01
96.88 0.22
90.24 0.03
94.78+ 0.04
95.38 0.08
94.21 0.05
79.24 0.05
80.21 + 0:07
SGSI
95.29+ 0.02
92. 92.76+ out
90.92++ 0.us
88, 88,17+ 0.64
93.25 0.84
88.06+ 0.03
79.05 0.05
77.90+ 0.12
SGSI w.Sparsity.
97.80 0.02
96.901 001
96.43.1 0.03
93.911 0.04
96. 06.261100 0.03
94.01: 0.02
82.92 0.02
81, $1.5441 0.02
SGSI w.In-deg.
97.42 0.02
96.82 0.2
95.90+ 0.u3
93.05+ 0.84
95.83+ 0.12
94.76+ 0.u2
80.80+ 0.03
80.34+ 0az
SGSI w. w.Out-deg.
97.43 0.02
96.801 0.02
95.87 0.03
93.011 1 0.04
95.62 0.02
94.87 1 0.02
81.05 0.03
80, 0.02
SGSI w. w.Both-deg.
97.90 0.02
96.95 0.03
96.46=
94.81 = IM83
96.32= 1083
95.00= 0.82
83.56 0.83
82.08 0.02
(3) SGSI w. Out-deg.: Similarly, each node's out-degree is guided by known limits or exact values. (4) SGSI w. Both- deg.: Combining in-degree and out-degree constraints for all nodes. The table shows average AUROC results of ten runs each on multiple datasets.
3000
Defaull
SGSI KI. Regn.
2500
SGSI Cloning
2000
Total
1500
As we see from the table, having sparsity typically raises final AUROC by 2-6% (e.g.. from 76% to 96 96.90% on VN.SP-30) reflecting how reducing unnecessary edges lets the model focus on essential interactions. Meanwhile, in-/out-dereee constraints further boost accuracy wherever domain rules about node connectivity align with the true graph. For instance, specifying in-degrees on VN_SP_51 increases AUROC from 90.92% to 95.90%. Enforcing both in- and out-degrees usually yields the top near-top scores; on VN VN_NS_50, performance jumps from 79.05% (uncon- strained) to 83.56%.
1000
500
200 400
600
800
1000
Epoch
Figure 4. Average loss curves for three variants of SGSI: Default SGSI (blue). SGSI without KL regularization of uncertain edges (SGSI w/o KL. Reg., red), and SGSI without cloning of the gating variables (SGSI w/o Cloning, green).
edges in the KL term (the red curve) forces the encoder to "spend bits" edges that are forcibly pinned, while not cloning gating (the green curve) disrupts stable gradient updates altogether. This aligns with the broader rationale behind VAE-based structural inference (c.g., ACD (Löwe et al., 2022), iSIDG (Wang & Pang, 2022), RCSI (Wang et al., 2023)), which the Variational Information Bottle- neck concept helps explain why prioritizing uncertain edges yields more effective adjacency discovery. Hence, these re- sults strongly validate the soft- goft-gating pipeline, in which both skipping known edges' KI. costs and cloning before clamp- ing are vital for stable optimization and maximal benefit from partial knowledge.
Overall, these results confirm that SGSI not only integrates partial adjacency knowledge on edge presence/absence but also extends smoothly to global sparsity and node-degree constraints. The method can gracefully deviate if data de- mands it, leading to state-of-the-arl performance across diverse domains and underscoring the flexibility of SGSI.
5.4. Ablation Studies
In Fig. 4, we show the average loss curves of SGSI and its two variants: without KL regularization of uncertain edges and without cloning of the gating variables, on VN_SP_50 dataset. As shown, Default SGSI converges to the lowest final loss with minimal oscillation, whereas failing to skip known edges in the KL or omitting the cloning step each lead to higher loss and greater instability. Moreover, both of the two variants yield lower or "matching" adjacency ac- curacy relative to each other and well below default SGSI's performance. From an IB perspective, not skipping known
5.5. Robustness to Inaccurate Prior Knowledge
SGSI leverages soft penalties and cloned gating to inte- grate partial knowledge while preserving flexibility. Be- cause these constraints are applied softly, via mild penalties and by clamping only cloned copy of the gating vector,
8

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Table Performance under Various Perturbations on VN_SP_100 and VN.NS_100. "Raw" denotes the unperturbed baseline. "F. 20% and "F, 50% known-present indicate that 20% and 50% of the prown-present edges (within a 30% prior knowledge set) are randomly flipped. respectively; similarly for "F. 20% known-absent and "F. 50% known-absent." "10% E. Sparsity" represent 10%/20%% errors on the global sparsity, while "10% E. In-Deg." and "10%/20% E, Out-Deg." denote errors on in-degree and out-degree estimates, respectively.
Raw
F. 20% KP
P. 50% KP
F. 20% KA
F. 50%0% KA
10% E. Sparsity
2016 E. Sparsity
10% E, In-Dag
201 E. In-Deg.
10% E. Our-Deg.
20% E. Our-Deg.
1000
88.17
90.03
89.70
91,45
90.16
92.61
91.08
92.70
90,94 78.18
92.66 79.80
90.82 78.23
VS 77.90
81,06
79.68
80.09
the model can deviate from incorrect priors when the data strongly contradicts them. In SGSI, the soft gating mask is applied after the Node-to-Edge operation (but not after Edge-to-Node), which helps preserve residual connectivity and prevents over-reliance on potentially flawed edges.
Impact Statement
This work's soft-gating approach to knowledge-awars struc- tural inference enables more trustworthy and transparent graph discovery by seamlessly integrating partial domain expertise. Beyond scientific modeling, it may influence fields like social network analysis or medical diagnostics, where partial relationships are known a prior. By clarifying latent structures in these sensitive domains, the technique has potential ethical and fostering safer, more explainable models. However, it also carries the societal consequence that misuse or over- reliance on domain knowledge could amplify biases in the data, underscoring the need for careful, context-informed deployment.
To further validate this, we conducted "noisy prior" experi- ments VN.SP IN.SP_100 and VN_NS_1ON 100, where randomly edges (with the overall prior knowledge set to 30%) and intro- duced 10% or 20% errors in global sparsity and degree constraints. Table 2 summarizes our preliminary results.
flipped 20% or 50% of the
spurious inferences
These results indicate that even with moderate noise, SGSI remains significantly more accurate than no-knowledge baseline with at least 1 2% margin. Although performance gains naturally diminish as noise increases, the model ro- bustly leverages available prior knowledge without catas- trophic failure. For more experimental results, including on real-world data, downstream tasks, and time complexity, please refer to Appendix E.
A Acknowledgements
The experiments presented in this paper were carried out us- ing the HPC facilities of the University of Luxembourg (Var- rette et al., 2022) (see hpc. uni -l... Besides that, author Jun Pang acknowledges financial support of the Institute for Advanced Studies of the University of Luxembourg through an Audacity Grant (AUDACITY-2021).
6. Conclusion
This paper introduces SGSI, a soft-gating approach for VAE- based structural inference that enables partial adjacency knowledge
edges, global sparsity and node degree constraints) to be integrated smoothly and without in-place gradient conflicts. Empirically, SGSI sur- passes existing baselines in structural fidelity, particularly at higher fractions of known edges. By skipping KL forcibly clamped edges and cloning before clamping, SGSI preserves stable gradient flow and effectively "frees capac- ity" for uncertain edges, whose insights align well with an information bottleneck perspective.
References
Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
Alemi, A. A., Fischer, I., Dillon, J. V., and Murphy, K. Deep variational information bottleneck. In Proceedings of the 5th International Conference Learning Representa- tions (ICLR), 2017.
Beyond relational discovery, these ideas of selectively ex- cluding known factors from a latent distribution can gener- alize to broader contexts. Potential future directions include dynamic graph inference, reinforcement learning where par- tial environment transitions are known, or even multimodal data fusion tasks such as integrating known sensor links with learned ones. In each domain, soft gating and care- ful KL management may unlock synergy between domain constraints and driven latent representations.
Alet, E. Weng, E., Jozano-Pérezz T., and Kaelbling, I. P. Neural relational inference with fast modular meta- learning. In Advances in Neural Information Processing Systems 32 (NeurIPS), 2019.
Bengio, Y., Lahlou, S., Deleu, T., Hu, J., Tiwari, M., and Bengio, E. Gflownet foundations. Journal of Machine Learning Research, 24(210):1 2023.

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Birkhoff, G. D. Dynamical systems, volume 9. American Mathematical Soc., 1927.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z.., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. Pytorch: An imperative style, -ee--pefforance deep learning library. Advances in Neural Information Processing Systems 32 (NeurIPS), 2019.
Brasó, G. and Leal-Taixé, L. Learning a neural solver for multiple object tracking. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 6247-6257, 2020.
Chen, C., Petty, K., Skabardonis, A., Varaiya, P., and Jia, Z. Freeway performance measurement system: mining loop detector data. Transportation Research Record, 1748(1): 96-102, 2001.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M.. Prettenhofer, P.. Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cour- napeau, D., Brucher, M., Perrot, M., and Duchesnay, Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Chen, S., Wang, J., and Li, G. Neural relational inference with efficient message passing mechanisms. In Proceed- ings of the 35th AAAI Conference on Artificial Intelli- gence (AAAI), pp. 7055-7063, 2021.
Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj. A., and
Chu. L.-F,, Leng, N., Zhang, J., Hou, Z., Mamott, D., Vereide, D. T., Choi, J., Kendziorski, C., Stewart, R., and Thomson, J. A. Single-cell ma-seq reveals novel regulators of human embryonic stem cell differentiation to definitive endoderm. Genome biology, 17:1-20, 2016.
Murali, T. Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nature Methods, 17(2):147 1544 2020.
Shalek, A. K., Satija, R., Shuga, J., Trombetta, J., Gennert, D., Lu, D., Chen, P., Gertner, R. S., Gaublomme, J. T., Yosef, N., et al. Single-cell ma-seq reveals dynamic paracrine control of cellular variation. Nature, 510(7505): 363-369, 2014.
Gu, A. and Dao, T. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752. 2023.
Ha, and Jeong, H. Unraveling hidden interactions in complex systems with deep learning. Scientific Reports, 11(1):1-13, 2021.
Smith, S. M., Miller, K. L., Salimi-Khorshidd. G., Webster, M., Beckmann, C. F., Nichols, T. E., Ramsey, J. D., and Woolrich, M. W. Network modelling methods for FMRI. Neuroimage, 54(2) 891, 201 la.
Hamilton, W., Ying, Z., and Leskovec, J. Inductive repre- sentation learning on large graphs. In Advances in Neural Information Processing Systems 30 (NIPS), 2017.
Smith, M., Miller, K. L.,
G., Webster, M., Beckmann, C. F. Nichols, T. E., Ramsey, J. D., and Woolrich, M. W. Network modelling methods for FMRL. Neuroimage, 54(2):875-891, 2011b.
Katok, A. and Hasselblatt, Introduction to the Modern Theory of Dynamical Systems. Encyclopedia of Mathe- matics and its Applications. Cambridge University Press, 1995.
Song, C.,, Lin, Y., Guo, S., and Wan, H. Spatial-temporal synchronous graph convolutional networks: A new frame- work for spatial-temporal network data forecasting. In Proceedings of the AAAI Conference Artificial Intelli- gence (AAAI). volume 34, pp. 914 921, 2020.
Kipf, T., Fetaya, E., Wang, K.-C., Welling, M., and Zemel, R. Neural relational inference for interacting systems. In Proceedings of the 35th International Conference on Machine Learning (ICML), pp. 2688-2697. PMLR, 2018.
Tishby, N. Pereira, F., and Biale, W. The information bottleneck method. In Proceedings of the 37th Annual Allerton Conference Communication, Control, and Computing (Allerton), pp. 368-377. IEEE, 1999.
Kwapieñ, J. and Droždž, S. Physical approach to complex systems. Physics Reports, 115(3))115-222) 2012.
Li, Ma, H., Zhang, 7., Li, J., and Tomizuka, M. Spatio- temporal graph dual-attention network for multi-agent prediction and tracking. IEEE Transactions on Intelligent Transportation Systems, 23()):0055611056 2022.
Tsubaki, M., Tomii, K., and Sese, J. Compound-prond interaction prediction with end-to-end learning of neural networks for graphs and sequences. Bioinformatics, 35 (2):309-318, 2019.
Löwe, S., Madras, D., Shilling, R. Z.., and Welling, M. Amortized causal discovery: Learning to infer causal graphs from time-series data. In Proceedings of the 1st Conference on Causal Learning and Reasoning (CLeaR), pp. 509-525. PMLR, 2022.
Varrette, S., Cartiaux. H., Peter, S., Kieffer, E., Valette, T., and Olloh, A. Management of an Academic HPC & Re- search Computing Facility: The ULHPC Experience 2.0. Proc. of the 6th ACM High Performance Computing
10

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
and Cluster Technologies Conf. (HPCCT 2022), Fuzhou, China, July 2022. Association for Computing Machinery (ACM). ISBN 978- 118-1-4500-9664--6
Wang, A. and Pang. Iterative structural inference of directed graphs. Advances in Neural Information Pro- cessing Systems 35 (NeurIPS), 2022.
Wang, A. and Pang, J. Active learning based structural inference. In Proceedings of the 40th International Con- ference on Machine Learning (ICML), pp. 36224-36245. PMLR, 2023.
Wang, A. and Pang. J. Structural inference of dynamical systems with conjoined state space models. Advances in Neural Information Processing Systems 37 (NeurIPS), 2024a.
Wang, A. and Pang, J. Structural inference with dynamics encoding and partial correlation coefficients. In Proceed- ings of the 12th International Conference on Learning Representations (ICLR), 2024b.
Wang, A., Tong, T. P., and Pang, J. Effective and efficient structural inference with reservoir computing. In Proceed- ings of the 40th International Conference on Machine Learning (ICML). pp. 36391-36660 1-36410. PMLR, 2023.
Wang, A., Tong, T. P., Mizera, A., and Pang, J. Benchmark- ing structural inference methods for interacting dynamical systems with synthetic data, The Thirty-eight Confer- ence on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
Webb, E., Day, B., Andres Terre, H., and Lió, Factorised neural relational inference for multi-interaction systems. arXiv preprints 10X.v:190..08721, 2019.
Wu, H., Liang Y., Xiong, W., Zhou, Z., Huang, W., Wang, S., and Wang. K. Earthfarsser: Versatile spatio-temporal dynamical systems modeling one model. Proceed- ings of the 38th AAAI Conference on Artificial Intelli- gence (AAAI), pp. 15906-15914 2024.

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Appendix of Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
A. More Details about SGSI
A.1. Sparsity Penalty
Alternative.
Besides the form Eqn. 11-12, we could also do an L2 penalty:
Lsparsity ε ae pE)
(22)
which more strongly penalizes large deviations. Either LI or L2 works in practice, though LI typically yields sparser solutions.
Combining with Known Edges. If a subset E+ of edges is forcibly clamped to ae 1, these edges are effectively non-negotiable. Since want the final fraction pE be addition these forced edges, do:
=a==
Σ c = Σ Oie
Espansity
Aeparsity
Σ ae - (pE - 18 |8+1|
(23)
unknown
EgE-
unknown
That ensures the known edges are not "counted" against the target fraction.
Practical Considerations. Besides the functioning and technical parts mentioned above, we provide practical instructions for the implementation of sparsity penalty:
A. Initialization: If gating parameters 0g are initialized around - 2.0 or -4.0, the initial o(0.) is near O. This already starts the network off with mostly "off" edges, which can help if we strongly suspect sparsity.
B. Tuning: The hyperparameter Asparsity is crucial. If see the final adjacency ignoring the fraction p. increase Asparsity If it prunes edges aggressively, decrease Asparsity-
The KL or IB penalty also encourages smaller gating uncertain edges. Combining the KL with an explicit global-sparsity penalty can lead to very pruned solutions, so please watch out for over-pruning if both 3 (KL weight) and Asparsity are large.
C. Influence on KL:
A.2. Node Degree Penalty
Maximum Degree Penalty. As we only mentioned the exact degree penalty in Section 4.2, it is feasible of setting up penalty for maximum node degrees. Suppose we want node to have out-degree at most h; We can define:
Laeg
Adeg
m0 outdeg(i) - ki)
(24)
This means as soon as outdeg(i) is above ki, we pay a cost proportional to the surplus. outdeg()) S k; penalty is zero. Similarly, for in-degree, we replace outdeg(i) with indeg(i).
Combining with Known Edges. node has some edges forcibly set (known present) (known absent), they naturally contribute to omit from the sum in outdeg (). That means:
Known Present edges from i + j each add to its out-degree. the domain says "these edges must exist," we might decide not to penalize them if they push out-degree above ky Alternatively, we still keep them the sum. If the node has to not exceed k; total, known edges reduce the "budget" for the uncertain edges.
Known Absent edges contribute 0. No penalty conflict arises there, since they do not increase out-degree.
12

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Practical Considerations. Besides the functioning and technical parts mentioned above, we provide practical instructions for the implementation of node degree penalty:
Tuning the Degree Penalty. Hyperparameter Adeg controls how strongly the network tries to meet node-degree constraints:
If Aeg is too low, the model may ignore the desired degree in favor of better reconstruction.
If Adeg is too high, risk overshadowing the data-driven eues, forcing all most nodes to match k even if that mismatches the observation signals.
We start with a small value (like 10-3 10 2) and see how the final out-degree distribution looks, adjusting from there.
Interplay with KL/IB: The IB KL cost also encourages compression, possibly making edges smaller fewer. Combining it with node-degree constraints might reinforce each other conflict if the data strongly wants more edges for certain nodes.
Interplay with Global Sparsity: We can have both a global L1 penalty (pushing overall fewer edges) and a node-degree penalty. Typically, each node's out-degree constraint is more specific, while the global penalty broad.
the node-degree sum across all nodes is contradictory the desired global fraction pE, the model will struggle. For instance, if each node is forced have out-degree 2 but p implies only 10% edges overall, yet 2N might exceed that fraction. Ensuring consistency in the constraints is crucial.
A.3. Implementation of SGSI
The overall pipeline of SGSI can be described in Algorithm 1. The implementation pseudocode of encoder can be found in Algorithm 2. The decoder of SGSI is same the MLPDecoder of NRI (Kipf al., 2018).
Algorithm 3 Two-Layer MLP with ELU and Batch Normalization
Require: Input dim nins hidden dim Thidd output dim Nout. dropout rate d
Ensure: Output tensor y with shape [batch, seq, nout]
I: Initialize parameters:
2: W1 < XavierNormal(Pnn. Thhia)
Layer weights
3: bj + 001
>> Layer 1 bias
4: W₂ < -XavierNorma(Mhia, nout)
Layer 2 weights
b2 +01.1
Layer 2 bias
? 1, B +
> Batch norm params
7: Forwardx
Input shape: [batch, seq, nin]
8: h + ELU (XWT + b1)
9: h < Dropout (h. d)
> Active only in training
10: - [hW, + b2)
11: h < BatchNorm (h, ()
> Sequence-wisg
12: Return: h
Algorithm Node-to-Edge Mapping with Gating
Require: Node embeddings E Nxxxxxd relation matrices Rice, Reendd B8 ExN
gating vector g E [0, 1
Ensure: Gated edge features Emssked Fx2d
1: Transpose V to V E BoxxxX
2: Expand R T
gend' R
to batch size 6
3: Compute S + V
x R
sgnd
Compute 4- VT
x R ree
Concatenate t- Concat(S, R.) along feature dim
6: Apply gating: Emasked
E O (g 2 124)
7: Return: Emakked
13

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Algorithm 1 Soft-Gated Structural Inference (SGSI) Training
Require: Dataset {X), known edges 8 (present), & (absent), node- seieededere//passtt constraints (optional), hyperparams
(3, Adeg, Asparsityy....)
Ensure: Learned parameters (6, 0) and final gating }
Initialize encoder parameters Φ, decoder parameters gating logits {0.].m
2: for each training iteration do
3: Sample mini-batch Xaath (x;) 1 from the dataset
4: (A) Encoder Forward
5: Compute 0. = fene (Xanch)) (e.g., via MLP or GNN)
6:
Obtain soft adjacency gating (size E): a 6 (0))
7:
Cloning/Clamping
8:
Otlamped f- a.clone()
9:
for e : aclemped [ t 1.0
10:
for = O'champed [ < 0.0
11:
Node Embedding: for each node j. h + c(1)
Sembod (Xg)
Node-to-Edge (Round 1): h.y fe (b. (1) D)
12:
13:
Gating 1: h C @cllmppelee]
14:
Edge-to-Node: h
(2) [o (
Node-to-Edge (Round 2): b2 (( (h+ 1)
15:
16:
Gating 2: :j hij (elampodie]]
(2) (2)
17:
Posterior Distribution: (Z X) softmax (e (Gumbel-Softmax)
(2)
18:
(B) Decoder Forward and Loss
19:
Use Oiclamped (or the distribution over Zui in a GNN decoder Xfuture + pe Pappaatate (batch Oclamped)
20:
Prediction Loss: Lpred =
log (Pftuuu Gclamped, Xbatc))
KI. Term: skip pinned edges, sum only over U: KT. Eeeu DKL. (no(Ee)) p(Ex))
22: Optional Constraints: Lsparsity = Asparsity [|accammeed 1, Ldeg = Adeg E, outdog(i) - ki]....
23: Total Loss: pred + 3 KL + Lsparsity + Laez
Backprop update (Ø, 0, {0.})
25:
end for
26: (C) Final Adjacency Extraction
27: Get adjacency from the Gumbel-SSfttax (Step 17)
28: return Learned parameters (ő, 0) and final adjacency
Algorithm 5 Edge-to-Node Aggregation with Gating
Require: Edge features [RpxxEx
relation matrix Riee E ExN
Ensure: Aggregated node features V Nxd
I: Transpose E to E RixdxE
2: Expand R to batch b: R E R
3: Aggregate: V agg + E R
> Shape: [b x d N]
4: Transpose Vage agg to [b x N x d
5: Compute in-degree: D >e=
E
Red [e
DE R
6: Normalize: V +- V age @ (D c))
O clement-wiss division
7: Return: V
The model of SGSI is implemented with PyTorch (Paszke et al., 2019), while Scikit-learn package was leveraged for the calculation of metrics (Pedregosa et al., 2011). Please refer to the attached link in supplementary mate- rials for the exact implementation. The implementation of SGSI is at: https: / http:::://g..tuu..oooo////////e SGST-Guides -tw--uuiidd-tttuttttrr--e--eeedd Iww.rralllleeeeeeeeeeee infference--leverrrgiinggroooo
com/ wwwp///gg/220033
Soft -tt--atinn--eecaane...
14

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Algorithm 2 Encoder of SGSI Forward Pass
Require: inputs of shape [b, N, i, cJ, adjacency index matrices rel rel,re, rel_send, known-present edges &+, known-absent edges -
Ensure: out of shape (6, E, nout] and gating E [0, 1
1: Flatten Inputs:
2: x + srpputs.vew((. N, -1) (shape: [b, N, features)
3: Node-level MLP:
4: x + mx) (shape: [b, NN masl}
5: Gating Initialization:
6: gating_raw < learnedAdj DeaarnedAdi.gatingg.....
7: gating a (gating raw raw.clone c))ne())
8: gating.clamped gating. gating.clone()
9: for = lore=1...6 E:
10: if (8, E E gating.clampedle] <- 1.0
11: else if (s.r) E E gating clamped[e] + 0.0
Node2Edge w/ Gating, MT.22:
edges gndie rel.re, relsend, gating.clamped (shape: [6, E, 2reid]}
14: edges +- mlp2(edges) {shape: [b, E, maal)
15:
x_skip +- edges
16: Factor Option (Edge2Node, MLP3, Node2Edge, MLP4):
17:
x_node + adge2hodeeggaiinn(ddgss rel_rec)
18:
node + mlp3(x_nool)
19:
edges +- uode2edge with gating( node, rel, rel send, gating gating.clampel clamped)
edges.2 2 cat(edges_2) 2, x_skip, dim 2)
21: odges +- mlp4(edges
22:
edges +- edges_2
23: Final Edge Logits:
24: out +- fc (edgee))s) (shape: [b, E, Nout]]
25: Return:
out, gating (Note that gating clamped used the message passing.)
A.4. Guidance of Hyperparameter Tuning
Besides the discussion about the hyperparameter values above, here we provide general guidance on how to perform search. For very sparse graphs (e.g., local interactions in physical systems), higher Aspariity values are tested. When node-degree constraints are precise (e.g., "exactly 2 neighbors"),. we set a higher Adeg. the prior knowledge is approximate, lower penalty weights prevent overshooting data evidence. These ranges avoid abrupt changes adjacency, ensuring that SGSI functions softly guided VAE rather than a rigidly constrained model.
B. Formal Statement of SGSI
In this section, we discuss why and how one skips the KL term for edges that are fully known present or absent concisely and formally. While we don't 't do an exhaustive derivation of every latent detail, this proof sketch clarifies why the KL can be omitted for clamped edges without contradicting the VAE's overall variational principle.
B. Setup
Let X the observed data (e.g., node trajectories), and Z the latent adjacency, with E potential edges total. A subset E of edges is preweepresent (definitely 1) and subset E of edges known-absent (definitely 0). The remaining uncertain edges form l {1,..., F.) \ (E+ UE-).
15

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
We assume the posterior (encoder) factorizes as:
go(Z
X)
96 EZz Z Zu
X)
II g0 Ze x) x 8 =
x =0, -,,
(25)
eEM
where 8()) denotes a delta distribution that fixes the edge to or 0 if belongs to E. E Similarly, we define the prior:
p(Z) = II P(z.) o(2 = -, e - x 8(2, = - EE8T),
(26)
cEu
assuming that the fully known edges are also considered "delta" under the prior, or equivalently that we have no uncertainty for those edges.
B.2. KL Divergence Factorization (Proof)
The KL divergence between the factorized posterior (Eqn. 19) and prior (Eqn. 20) is:
DKL (Z x))lp(())
Σ DKL (go (ee XX) [z())))
DKL
eEU
since (ōll) = 0 for known edges. This shows that only uncertain edges contribute to the compression term T(X; Z).
B.3. Interpretation: Skipping the KL for Known Edges
In practice, we omit pinned edges from the encoder's 's random portion, or equivalently set their posterior distribution to a 5. Because (oll) = O, no bits are required to store these known edges. Only the uncertain edges in V appear in the KI. summation:
DKL (g X) [lp(Z)) = Σ DKL DRI(10 (Ze X) ( pl.))
(27)
eEW
This formalizes why forcibly pinned edges do not need a random distribution or KL cost.
In conclusion, by delta-encoding fully known edges in both posterior and prior, the KL for those edges is zero. Implementing this in code means we skip them in the KI. summation. The uncertain edges remain random, paying usual KI. cost. This ensures:
1. No Contradictory Signals:
The pinned edges do not conflict with prior-latent assumption of randomness.
2. Freed Capacity: The model invests "bits" only in uncertain edges, aligning with the information bottleneck goal of minimal yet sufficient adjacency.
Hence, skipping known edges' KL is justified by the factorization of d-like edges in the posterior and prior.
C. More Details about Datasets
In this section, we provide more details about the datasets used this work apart the description in Section
C.1. Springs Simulations
To generate these Springs Simulations datasets, we follow the description of the data in (Kipf et al., 2018) but with fixed connections and with 10 nodes, in order to simulate spring-connecte particles' motion in a 2D box using the Springs simulation. In this setup, nodes represent particles, and edges correspond to springs governed by Hooke's law. The Springs simulation's 's dynamics are described by a second-order ordinary differential equation: m; - xt Ejex, -k (z:(t) (t). Here, m; represents particle mass (assumed 1), k is the fixed spring constant (set to 1), and N₂ is the set of neighboring nodes with directed connections to node i, which is sub-sampled from the graphs generated in the StructInfer in previous steps. We integrate this equation to compute (t) and subsequently (t) for each time step t. The resulting values of (t) and (t) create 4D node features at each time step. To be specific, at the beginning of the data generation for each springs dataset, we randomly generate a ground truth graph and then simulate 12000 trajectories on the same ground truth graph, but with different initial conditions. The rest settings are the same as that mentioned in (Kipf et al.,
16

---

Guided Structural Inferenee: Leveraging Priors with Soft Gating Mechanisms
2018). We collect the trajectories and randomly group them into three sets for training, validation and testing with the ratio of 8: 2: 2, respectively.
C.2. NetSims
It is firstly mentioned in (Smith al., 201 la), which offers simulations of blood-owygen--eeel-deeeeeeee (BOLD) imaging data in various human brain regions. Nodes in the dataset represent spatial regions of interest from brain atlases functional tasks. Interaction graphs from the previous section determine connections between these regions. Dynamics are governed by a first-order ODE model: (t) = a EjEN, Zz(t) - a (t) + C Ug where σ controls temporal smoothing and neural lag (set to 0.1 based on (Smith et al., 201 la), and C' regulates external input interactions (set to zero to minimize external input noise) (Smith et al., Ia). ID node features at each time step are obtained from the sampled (t)...
C.3. Synthetic Biological Networks
The six directed Boolean networks (LI, LL, CY, BF, TE, BF-CV) are the most often observed fragments in many gene regulatory networks, each has 7, 18, 6, 7, 8 and 10 nodes, respectively. Thus by carrying out experiments on these networks, we can acknowledge the performance of the chosen methods on the structural inference of real-world biological networks. We collect the six ground-truth directed Boolean networks from (Pratapa et al., 2020) and simulate the single-cell evolving trajectories with BoolODE (Pratapa al., 2020) (https: / /tgtttb.b.../gggggg settings mentioned in that paper for every network. We first sample a total number of 12000 raw trajectories. We then sample different numbers of trajectories from raw trajectories and randomly group them into three datasets: for training, for validation, and for testing, with a ratio of 8 : 2 2. After that, we sample different numbers of snapshots according to the requirements of experiments in Section 5.1 with equal time intervals in every trajectory and save them as ".npy files for data loading.
/ooMMrralll i group/ Bool ODE) with default
C.4. StructInfer Benchmark
The StructInfer benchmark (Wang et al., 2024) evaluated 12 structural inference methods in a comprehensive way on a synthetic dataset. The dataset covers 11 types of different underlying interaction graphs and two types of dynamical simulations. (https://structinn) www.iineeerggguiiie
io/) As there are so many trajectories, we chose the ones under the name 'Vascular Networks', or in short 'VN', whose underlying interaction graphs approximate the real-world vascular networks in biology systems. As the data is already split into three sets: for training, for validation, and for testing, we keep this setting. In the following paragraphs, we describe more details about the Springs and NetSims simulations utilized by the StructInfer benchmark.
For Springs simulation, it follows the approach by (Kipf et al., 2018), to simulate spring-con crring-connected particles' motion in a 2D box using the Springs simulation. this setup, nodes represent particles, and edges correspond to springs governed by Hooke's law. But different from Springs Simulations mentioned above, StructInfer generates ground-trute truth interaction graphs with the graph properties of the real-world graphs network. The ground-truth interaction graphs used to determine the connectivity between the nodes. The Springs simulation's 's dynamics are described by a second-order ordinary differential equation: my xx (4) Ejex; k. (z.(() - =1) Here, mı represents particle mass (assumed as 1), k is the fixed spring constant (set to 1). and Nr is the set of neighboring nodes with directed connections to node i, which is sub-sampled from the graphs generated in the StructInfer in previous steps. We integrate this equation to compute (t) and subsequently (t) for each time step l. The resulting values of (t) and (i) create 4D node features at each time step.
For NetSims simulation, it is firstly mentioned in NetSim dataset (Smith et al., 201 which offers simulations of blood- oxygen-level seren-leeelldeeddeen (BOLD) imaging data in various human brain regions. Nodes in the dataset represent spatial regions of interest from brain atlases or functional tasks. But different from NeiSim mentioned above. StructInfer generates -ouun interaction graphs with the graph properties of the real-world graphs or network. The ground-truth interaction graphs are used to determine the connectivity between the nodes. Dynamics are governed by first-order ODE model: <t())= = o. JEN, (t) a (t) + C- u4 where a controls temporal smoothing and neural lag (set to 0.1 based on (Smith et al., 201 la), C regulates external input interactions (set to zero to minimize external input noise) (Smith et al., 2011a) ID node features at each time step are obtained from the sampled (t)..
17

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
D. More Details about Baselines
For the experiments without prior knowledge, we follow the official implementation of the baselines. As for the integrating of the prior knowledge, we leverage different strategies. For the methods based on VAEs, (e.g. NRL MPM, ACD, iSIDG, RCSI), we directly perform overwriting of latent variables corresponding known edges, while keep the rest following the original implementation.
NRI
NRI (Kipf et al., 2018) a VAE-based model for unsupervised relational inference. We use the official implementation code by the author from https: / thub. com/ / fetaaa aaaaaaaaaE... /NRI with customized data loader for chosen datasets. We add our metric evaluation in the 'test' function, after the calculation of accuracy in the original code. Besides, after carefully comparison, we used the NRI version with sparsity regularization, yielding higher performance, which was also implemented by the authors.
D.2. MPM
MPM (Chen et al., 2021) employs VAE framework with a relational interaction mechanism and spatio-temporal mes- sage passing. We use the official implementation code by the author from https: i thub. com/hi h lbe r 9221 / NRI-MPM with a customized data loader for our chosen datasets. add our metric evaluation for AUROC in the "evallate()' function class "XNRIDECIns" in the original code.
D.3. ACD
ACD (Löwe et al., 2022) utilizes shared dynamics to infer causal relations within datasets. We follow the official implementation code by the author as the framework for ACD (https:/ / com/ 1oewex/ mmmrrttzddauaaaiecooerr
We run the code with customized data loader for the datasets in this work. We implement the metric-caleulation pipeline in the 'forward pass and eval()" function.
D.4. ISIDG
iSIDG (Wang Pang, 2022) iteratively refines adjacency matrices enhance directional inference. We follow the official implementation code by the author the framework for iSIDG (https://githhu) com/wang422003/ wtwpr//gggsstttttttaaa--eeeeeeee ateee//mann/srrcc/mmel////s/ss//
-orecce---eooo---o--------eeee Interatt ing-Dynmmicall Sysllmss//m////// We disable the metric evaluations for the AUPRC and Jaccard index in the original implementation of iSIDG for faster computation.
D.5. RCSI
RCSI (Wang et al., 2023) integrates reservoir computing for efficient structural inference. We follow the offi- cial implementation code by the author as the framework for RCSI (https: com Darer-rooontterrttttoo---oaaaaaa http:///wwwwwww//////0000 wang422003 Systtmas/ Same as iSIDG, we disable the metric evaluations for AUPRC and Jaccard index in the original implementation RCSI for faster computation.
rree
D.6. ALaSI
ALaSI (Wang& & Pang, 2023) leverages deep active learning for scalable structural inference, We thank the authors for communicating and providing the code. We use the evaluation metries implemented by the authors.
D.7. SICSM
SICSM (Wang & Pang. 2024a) leverages MAMBA(Gu & Dao, 2023) and generative flow network (Bengio et al., 2023) for structural inference on irregularly sampled trajectories, but can still work on more broader case like uniformly sampled data. We thank the authors for communicating and providing the code, which is based on PyTorch and diverges from the implementation mentioned in their paper. We use the evaluation metrics implemented by the authors.
18

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
E. More Details about Experiments
E.1. More General Settings
All experiments mentioned in Section were conducted on a single NVIDIA Ampere 40GB HBM graphics card, paired with AMD Rome CPUs (32 cores @2.3 GHz). During training. we set batch size as 128 for datasets which have less than 30 nodes, for those having 30 50 nodes, we set batch size as 64. For the rest of the data sets, set the batch size be 16. The learning rate is set to be x 10-4 We train SGSI model with 1000 epochs on every dataset.
The choice of the hyperparameters in the loss function play a non-neglect role in training SGSI, and the their values are searched via Bayesian Optimization toolbox Optuna (Akiba et al., 2019). We set the bounds for 5, Asparsity and Adeg as [1.0, 2.5], [10-3, 10-32,, and [10-1, 10-2), respectively. The values the hyper-parameter are summarized in Table 3.
Table 3. Hyper parameter choices for every dataset.
DATASET
Japaraity
Adeg
Springs NetSims
0.011
5 X 10
0.008
5 10
LI
0.005
10-2
LL
1.1
0.008
5 10-3
0.005
10 2
BF
0.005
10-2
TF
0.005
10-2
BF-CV
0.006
10-2
VN_SP_15
1.1
0.005
10-2
VN_SP_30
1.3
0.004
10-2
VN_SP_50
1.5
0.004
10-2
VN SP 100
1.8
0.003
10-2
VN NS 15
1.1
0.005
10-2
VN_NS 30
0.004
10-2
VN
1.5
0.004
10-2
VN_NS_100
1.8
0.003
10-2
E.2. Experimental Results PEMS
In addition the data sets mentioned in Section we selected another sets of datasets, which are derived from the Califomna Caltrans Performance Measurement System (PeMS) (Chen et al., 2001), comprise data aggregated into 5-minute intervals. The adjacency matrix of the nodes is constructed by the distance of the road network with Gaussian kernel thresholded (Song et al., 2020). Table 4 summarizes these datasets. We resampled the data such that constructing 49 time
Table 4. Statistics of PEMS datasets.
Dataset
# Nodes
Edges
Time Steps
Missing Ratio
PEMS03
358
547
26, 208
0.672%
PEMS04
307
340
16, 992
3.182%
PEMS07
883
866
28, 224
0.452%
steps of points for each trajectory, and obtained 12000 trajectories for each with overlapping smapshots. It's important to note that these datasets' adjacency matrices only connect sensors on the same road, omitting alternative connecting paths, which could impact results.
After that, we investigate onto the performance of SGSI having 20% of known present edges, 20% of known absent edges, global sparsity and node degree constraints (including in-degree and out-degree) as prior knowledge, respectively. In order to run on these datasets with more than 300 nodes, we use mini-batching technique similar to GraphSAGE (Hamilton et al., 2017). Yet we have to utilize 5-10 GPUs to train SGSI on PEMS datasets. We report the average AUROC results of ten runs Table 5.
19

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Table 5, Average AUROC results (%) PEMS datasets.
Prior Knowledge
Datasct
No Prior
20% Kn. Present
20% Kn. Absent
Sparsity
In- De-rree
Out-Degree
In-/Out-Degree
PEMS03
7
77.15: 0.05
76.33+
76.85 0.05
76.06 0.05
76.07 0.06
77.01
PEMS04
74.20 0.09
79.07 0.06
78. 78.633t 0.07
75.38 0.06
77.52 ::2::000 0.06
77.54+ 0.06
77.98: 0.06
PEMS07
73.94 0.07
76.18 0.09
75.62+ 0.08
75.54 0.08
75.03 0.05
75.04 0.04
75.12 0.01
Table 6. Average training time (in hour) of ten runs of SGSI and baseline methods on VN_NS datasets.
Methods
VN NS VN.NS 30
VN VN.NS. 50
VN NS 100
NRI
24.5
33.5
40.6
47.2
MPM
45.3
60.4
79.2
83.6
ACD
40.1
53.1
67.6
81.6
iSIDG
43.8
56.0
88.5
97.8
RCSI
44.5
57.8
91.7
102.8
ALaSI
25.0
26.8
27.9
46.2
SICSM
56.3
69.2
91.2
117.1
SGSI
24.6
33.5
40.5
47.1
Across all PEMS datasets, embedding prior knowledge (20% known edges or constraints) improves over the "No Prior" baseline. For instance, on PEMS03, the AUROC rises from 70.82% to around 76-77%, and on PEMS04 from 74.20% to roughly 78-79%. Known Present typically yields slightly higher final AUROC than Known Absent (e.g.. PEMS03: 77.15% vs. 76.33%) which suggests that confidently foreing certain edges to helps SGSI focus on the uncertain regions more effectively. By contrast, clamping edges to can eliminate potential connections but still offers some benefit over no prior knowledge. Applying sparsity penalty sometimes yields moderate improvements, but does not always outperform known-edge constraints. On PEMS04, for example, Caparsity leads 75.38%, which is less than the near 78-79% range seen for known-presenne peoow-ppreeen/absent edges. This indicates that while encouraging fewer total edges helps, the advantage smaller than providing explicit knowledge of which edges exist do not. Enforcing in-degree or out-degree alone yields modest improvements (PEMS04 from 74.20% to 77.52-77.55 -7.54)7. Combining both in- and out-degree is particularly helpful on PEMS03 (77.01 % vs. 76% for each alone), though on PFMS07, the difference is smaller (75.12% vs. 75.03-75.04%%
In summary, Table demonstrates that SGSI meaningfully improves adjacency recovery when equipped with partial knowledge or constraints, even for mid-sized road sensor networks like PEMS. The interplay of known edges, node-degree constraints, and a potentially incomplete adjacency can all influence final performance, but partial knowledge nearly always raises AUROC beyond the no-prior baseline.
E.3. Training Time Comparison
Table 6 presents the average training time (in hours) for runs of each method on the VN_NS datasets, as the number of nodes grows from 15 to 100. A surprising standout is ALaSI, requiring only 25 28 hours on node sets of size 15 15-50, which is markedly faster than other baselines of comparable complexity (e.g., MPM, ACD, iSIDG). Even the largest dataset (VN-NS_100 100, ALaSI takes 46.2 hours, on par with SGSI at 47.1 hours and below many baselines exceeding 80-100 hours. This suggests that ALaSSIs active learning approach may yield high computational efficiency for moderate node counts. NRI and SGSI remain quite close in run times from VN.NS.15 (24.5 vs. 24.6 hours) up to VN_NS_100 (47.2 vs. 47.1 hours). This indicates that soft gating and partial-knowledge integration not add major overhead to the NRI pipeline, keeping SGSI's training cost roughly equivalent NRI's. Methods such as MPM. ACD, iSIDG, RCSI, and SICSM typically require significantly more training time, especially at node sizes beyond 30 or 50. For instance, iSIDG and RCSI each exceed 90 hours on VN_NS_50 and surpass 97-100 hours on VN_NS_100 while SICSM scales the slowest (56-117 hours). These results imply that both SGSI and AI ALaSI maintain strong scalability, with SGSI's gating and KL-skipping design incurring little additional cost compared to NRI. By contrast, iterative refinement or heavier GFlowNet sub-space expansions (e.g.. iSIDG, RCSI, SICSM) show more pronounced slowdowns as node counts climb.
Besides, we provide the training-tii comparison across our main datasets (Springs, NetSims, LI, LL,, and the 100-node
20

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Table TTraiinnnn Comparison (in hours) across Main Datasets for SGST with 20% known-present edges, NRI, and iSDDG.
Methods
Springs
NetSims
LI
LL
VN SP-0P0000
VN NS 100
NRI
20.1
16.0
14.3
18.2
49.0
47.2
iSIDG
42.2
36.9
48.1
50.6
100.6
97.8
SGSI
20.4
15.6
14.7
18.1
49.2
47.1
Table 8. A AUROC values toy dataset with 5k nodes generated from Springs Simulations.
K.P. 10%
K.P. 20%
K.P. 30%
K.A. 10%
K.A. 20%
30%%
Spar.
w. In-deg.
Out-deg.
w. Both-deg.
SGSI
0.56
2.40
4.93
0.77
3.08
5.73
7.46
6.90
7.13
7.68
VN datasets) and show results in Table SGSI consistently matches NRI's training time (e.g., 20.4h vs. 20.1h on Spring Simulations and 47.1h vs. 47.2h for VN NS 100), while methods such iSIDG require 1.5-2x longer. Yet recall that SGSI achieves 79.7% AUROC on Springs, which is higher than NRI. Overall, these results confirm that SGSI's partial-knowledy gating imposes minimal additional cost while providing meaningful accuracy gains in structural inference.
Overall, SGSI and ALaSI stand out for balancing competitive speed with strong adjacency inference performance, making them appealing for larger or more demanding relational inference tasks.
E.4. Scalability to Larger, Heterogeneous Graphs
SGST is designed to scale via mini-batching and subgraph sampling, approaches similar to GraphSAGE, which allow SGSI's gating parameters to be computed or stored per subgraph rather than across the full edge space. For extremely large graphs, we can either (i) restrict gating local neighborhoods, or (ii) compute gating logits on the fly from node embeddings, thereby avoiding parameter explosion.
In this paper, we demonstrate scalability the PEMS datasets (approximately 300 nodes) by leveraging multi-GPU mini-batching (in Appendix E.2). To further validate SGSI on larger graphs, we generated a toy dataset with 5,000 nodes using Springs Simulations. Table 8 shows the AUROC values under various levels of prior knowledge. These results indicate that SGSI, when using mini-batching, remains effective even at the 5k-node scale with prior knowledge, and with meaningful improvements in AUROC from 0.56 to 7.68.
SGSI's flexible architecture inherently supports heterogeneous graphs. By adopting per-relation gating parameters and applying the clone-and-clamp strategy each edge type, SGSI can differentiate among various relations and node types while maintaining stable gradient flow and effective KL skipping. Although our current domain primarily involves homogeneous datasets, we recognize SGSI's potential in applications such as multi-modal social networks, biomedical systems, and multi-layer transportation networks, and plan to explore these in future work.
E.5. Prior Knowledge in Real-world Scenarios
In order to validate SGSI with prior knowledge under real-world scenarios, we' 've conducted additional experiments on real-worid single-cell RNA-seq datasets: (1) hESC (human embryonic stem cells) (Chu et al., 2016) and (2) mDC (mouse dendritic cells) (Shalek et al., 2014). These biological systems represent scenarios where reliable prior knowledge, such as known interactions or absent regulatory connections, often exists due to extensive experimental validations in literature.
In Table 9, we summarize the AUROC (%) of SGSI without prior knowledge (SGSI (Raw)), along with SGSI leveraging partial prior knowledge (10% 15% known-present (K.P.) and known-absent (K.A.) edges).
From these results, even modest incorporation (10- 15%) of experimentally validated knowledge consistently improves structural inference, demonstrating that SGSI can practically leverage partial prior knowledge in real biological settings. While absolute improvements may appear modest, such incremental gains are significant in real-world domains like biology, where even slight improvements in inferred networks can lead to more meaningful biological interpretations and robust downstream analyses.
21

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
Table AUROC values real-world datasets with different prior knowledge.
hESC(AUROC %)
mDC(AUROC %)
SGSI (Raw)
50.18
52.63
SGSI + 10% K.P.
51.35
53.91
SGSI + 15% K.P.
51.70
54.14
SGSI + 10% K.A.
51.41
53,96
SGSI + 15% K.A.
52.26
54.65
Table 10. Mean Squared Error (MSF) for 10-Step Future Prediction on Springs Simulations and VN_SP/VN_NS Datasets. We compare NRL, SGSI without prior knowledge (raw), and SGSI with 20% known-present edges.
Springs
VN_SP_R _30
VN_SP_50
VN.NS_30
VN.NS_50
NRI SGSI raw
5.78e-6
3.24e-5
7.81e-5
4.08e-4
1.20e-3
5.77e-6
3.26e-5
7.73e-5
4.06e-4
1.17e-3
SGSI W. Prior
4.92c-6
2.01e-5
6.13e-5
1.83e-4
9.60e-4
E.6. Evaluation on Downstream Tasks
Although our primary goal is to infer the underlying interacting structure of dynamical systems, SGSI is also applicable to time-series forecasting. For instance, we report Mean Squared Error (MSE) for predicting future steps on several datasets. The results are shown in Table 10. We compare NRI, SGSI without prior knowledge (SGSI raw), and SGSI with 20% known present edges.
Our results demonstrate that incorporating partial prior knowledge (SGSI with 20% known-present edges) significantly enhances predictive accuracy compared to both SGSI without prior knowledge and the established baseline method NRI. For example, on the Springs Simulations dataset, SGSI with prior knowledge reduces the MSE to 4.92 x 10 from 5.78 x 10-6 achieved by NRI. Similar consistent improvements are observed on VN SP and VN NS datasets, confirming that leveraging domain knowledge about known connections helps the model make more precise future state predictions.
These findings reinforce the value of our soft-gating mechanism, highlighting that even modest incorporation of known edges substantially enhances model performance beyond structural inference alone. This indicates the broader applicability and practical relevance of SGSI in realistic predictive scenarios.
F. Limitations of SGSI
Despite its advantages integrating partial knowledge and providing flexible structural inference, SGSI is subject to several limitations:
Dependence on Accurate Prior Knowledge. SGSI heavily relies on the quality of user-specified constraints (e.g.. pooon-present/bbsett edges, node degrees). If these constraints are inconsistent with the true system or contain significant errors, the gating mechanism may learn compromised adjacency. In some domains, partial knowledge itself might be limited or noisy, negating the expected accuracy gains.
Complex Constraint Setup. Incorporating domain knowledge (e.g., specifying exact node degrees or defining known edges) requires substantial manual effort or domain expertise. If a user cannot confidently specify these constraints, SGSI defaults to purely data-driven approach, which amounts to standard VAE-based inference. Consequently, the method's performance improvements become less pronounced or revert to baseline levels.
Scalability for Very Large Graphs. Although SGSI can scale better than naive adjacency enumeration by skipping or clamping known edges, extremely large graphs (e.g.. hundreds thousands of nodes) still pose significant computational challenges. In such scenarios, subgraph sampling or approximate adjacency factorization is necessary; implementing these strategies demands additional code modifications (e-g., bati-bbatching or sparse message passing), potentially complicating
22

---

Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms
the pipeline.
Additional Hyperparameter Tuning. Soft gating and constraint integration introduce extra hyperparameters (Aaparsity> Adeg. etc.) beyond the typical VAE setting. Finding suitable values for these penalties often requires small grid search or heuristic tuning. This overhead can be minor, but in practice may complicate experimentation.
Despite these drawbacks, SGSI remains a robust framework for knowledge-aware structural inference, offering strong performance boosts where partial adjacency constraints are reliable and computational resources allow. Future improvements may target adaptive constraint weighting, more automated constraint specification, and extended scalability mechanisms for extremely large densely connected graphs.
23


Reviews:
[{"review_id": "c5fGMKViEz", "reviewer": "Reviewer_vBqs", "summary": "The paper introduces SGSI, a framework for latent graph structure learning that integrates partial prior knowledge into a VAE. It employs a soft gating mechanism with learnable parameters to smoothly control edge activation and uses a cloning and clamping strategy to fix known-present and known-absent edges without disrupting gradient flow. By enforcing adaptive regularization for global sparsity and node-degree constraints, SGSI effectively separates known from uncertain edges, optimizing the trade-off between compression and prediction via an information bottleneck perspective. Empirical results on diverse datasets—including physical simulations, biological networks, and multi-agent systems—demonstrate up to a 9% AUROC improvement over existing methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your positive assessment and valuable comments. In response, we clarify our approach as follows (tables can be found at https://anonymous.4open.science/r/SGSI-Rebuttal-1614/Tables.pdf .)\n\n**1. Hyperparameters**\n\nSGSI introduces hyperparameters, most notably, the KL weight $\\beta$, the sparsity penalty$ \\lambda_{\\mathrm{sparsity}}$, and the node-degree penalty $\\lambda_{\\mathrm{deg}}$. These parameters are essential for balancing the trade-off between inference and the incorporation of prior knowledge. We conducted Bayesian optimization with systematic grid searches on a validation subset to identify a “reasonable zone” that yields both high AUROC and stable convergence. Our initial search considered:\n\n- $\\beta \\in \\\\{0.1, 0.5, 1.0, 1.3, 1.4, 2.0, \\dots\\\\}$,\n- $\\lambda_{\\mathrm{sparsity}} \\in \\\\{10^{-4}, 10^{-3}, 10^{-2}\\\\}$,\n- $\\lambda_{\\mathrm{deg}} \\in \\\\{10^{-4}, 10^{-3}, 10^{-2}\\\\}$.\n\nOur domain heuristics further guide these choices:\n\n- When the underlying graph is known to be very sparse (e.g., physical systems with local interactions), a larger $\\lambda_{\\mathrm{sparsity}}$ is preferable.\n- When node-degree constraints are well-established (e.g., each node has exactly 2 neighbors), a higher $\\lambda_{\\mathrm{deg}}$ ensures the model respects these constraints.\n- Conversely, if the prior knowledge is only approximate, lower penalty weights prevent the model from being over-constrained.\n\nWe showcase results of sensitivity study on the VN_SP_30 dataset, and the results can be found in the Table 1 in the attached link. The results demonstrate that the optimal performance is achieved with $\\beta=1.3, \\lambda_{\\mathrm{sparsity}}=0.004$, and $\\lambda_{\\mathrm{deg}}=0.01$, yielding an AUROC of 92.76%. Variations from these values result in a modest decrease in performance, which validates that while SGSI is indeed sensitive to these hyperparameters, it operates robustly within a reasonable range.\n\n**2. Robustness to Inaccurate Prior Knowledge** \n\nSGSI leverages soft penalties and cloned gating to integrate partial knowledge while preserving flexibility. Because these constraints are applied softly, via mild penalties and by clamping only a cloned copy of the gating vector, the model can deviate from incorrect priors when the data strongly contradicts them. In SGSI, the soft gating mask is applied after the Node-to-Edge operation (but not after Edge-to-Node), which helps preserve residual connectivity and prevents over-reliance on potentially flawed edges.\n\nTo further validate this, we conducted “noisy prior” experiments on VN_SP_100 and VN_NS_100, where we randomly flipped 20% or 50% of the known-present/absent edges (with the overall prior knowledge set to 30%) and introduced 10% or 20% errors in global sparsity and degree constraints. Table 2 in the attached link summarizes our preliminary results.\n\nThese results indicate that even with moderate noise, SGSI remains significantly more accurate than a no-knowledge baseline with at least 1~2% margin. Although performance gains naturally diminish as noise increases, the model robustly leverages available prior knowledge without catastrophic failure. \n\n**3. Scalability to Larger, Heterogeneous Graphs**\n\nWe appreciate your interest in extending SGSI to larger or heterogeneous graphs. SGSI is designed to scale via mini-batching and subgraph sampling, approaches similar to GraphSAGE, which allow SGSI’s gating parameters to be computed or stored per subgraph rather than across the full $N \\times N$ edge space. For extremely large graphs, we can either (i) restrict gating to local neighborhoods, or (ii) compute gating logits on the fly from node embeddings, thereby avoiding an $\\mathcal{O}(N^2)$ parameter explosion.\n\nIn this paper, we demonstrate scalability on the PEMS datasets (approximately 300 nodes) by leveraging multi-GPU mini-batching (in  Appendix E.2). To further validate SGSI on larger graphs, we generated a toy dataset with 5,000 nodes using Springs Simulations. Table 3 in the attached link shows the $\\Delta$AUROC values under various levels of prior knowledge. These results indicate that SGSI, when using mini-batching, remains effective even at the 5k-node scale with prior knowledge, and with meaningful improvements in $\\Delta$AUROC from 0.56 to 7.68.\n\nSGSI’s flexible architecture inherently supports heterogeneous graphs. By adopting per-relation gating parameters $\\theta_{e,r}$ and applying the clone-and-clamp strategy to each edge type, SGSI can differentiate among various relations and node types while maintaining stable gradient flow and effective KL skipping. Although our current domain primarily involves homogeneous datasets, we recognize SGSI’s potential in applications such as multi-modal social networks, biomedical systems, and multi-layer transportation networks, and plan to explore these in future work.\n\nThank you again for your positive feedback. We look forward to refining our manuscript with these additional experiments.", "claims_and_evidence": "The paper’s main claims are generally well supported by both theoretical insights and empirical results.", "methods_and_evaluation_criteria": "I think they make sense.", "theoretical_claims": "There are theorem statements in the paper.", "experimental_designs_or_analyses": "The experimental design is generally sound. The authors evaluate SGSI on several benchmark datasets—including Spring Simulations, NetSim, synthetic biological networks, and vascular networks—and compare its performance against multiple baselines using AUROC as a metric. They vary the fraction of known-present and known-absent edges, which effectively demonstrates how partial prior knowledge improves inference accuracy. In addition, the paper includes detailed ablation studies (e.g., omitting the KL regularization on known edges or skipping the cloning step) to isolate the impact of each component on training stability and performance.", "relation_to_broader_scientific_literature": "It builds on VAE-based structural inference methods (e.g., NRI by Kipf et al., 2018; Alet et al., 2019; Chen et al., 2021) that aim to learn latent graph structures, but goes further by incorporating partial prior knowledge—something earlier approaches tend to overlook.\n\nThe use of soft gating to control edge activation relates to established techniques in neural network regularization and gating mechanisms, ensuring smooth gradient flow compared to naive overwriting methods used in some prior works.", "essential_references_not_discussed": "They include many references. It seems quite comprehensive in general.", "other_strengths_and_weaknesses": "Strengths include its originality in integrating soft gating, cloning/clamping, and adaptive regularization to merge domain knowledge with latent structure learning. The paper is also theoretically grounded through an Information Bottleneck perspective and validated by strong empirical results. \n\nWeaknesses involve limited analysis on scalability to large or heterogeneous graphs, modest hyperparameter sensitivity studies, and reliance on the availability and accuracy of prior knowledge.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. How robust is SGSI when the prior knowledge is noisy or partially inaccurate? How this will affect the approach proposed?\n\n2. Can you provide additional experiments or analysis on the scalability of SGSI to larger and more heterogeneous graphs? A positive response with empirical evidence or theoretical insights would strengthen the scalability claim and improve my evaluation.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "el9CiDHJn9", "reviewer": "Reviewer_4t2y", "summary": "The paper \"Guided Structural Inference: Leveraging Priors with Soft Gating Mechanisms\" introduces Soft-Gated Structural Inference (SGSI), a variational autoencoder (VAE)-based method for inferring latent relational structures while integrating domain constraints.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your constructive feedback regarding our paper SGSI. Below, we address your concerns point by point:\n\n**1. Marginal Gains on Certain Datasets**\n\nWhile SGSI shows up to 9% AUROC improvement on some datasets, other cases yield more modest gains (1–2%). We believe this reflects two factors: (i) the inherent difficulty or limited availability of reliable partial knowledge in those tasks, and (ii) that some baselines (e.g., in Spring Simulations) already achieve high AUROC, leaving limited room for further improvement. Importantly, even small improvements can be significant for domain practitioners, especially given SGSI’s additional benefits in interpretability (via clamped edges) and stable gradient flow. Moreover, when no prior knowledge is available, SGSI effectively reverts to a standard VAE-based approach (similar to NRI), explaining the smaller gains in those scenarios.\nWe believe this variation reflects two factors:\n\n1. **Domain Impact:**\n\nIn domains where partial prior knowledge is available, such as transportation networks or biomedical systems, a substantial gain (up to 9% AUROC) can lead to significantly improved decision-making and reliability. For example, in critical applications like traffic management or clinical diagnostics, a 9% improvement in edge prediction accuracy can meaningfully enhance system interpretability and safety.\n\n2. **Baseline Ceiling Effects and Additional Benefits:**\n\nIn some tasks like Spring Simulations, the baselines already achieve high AUROC scores, leaving little room for large numerical gains. Even when improvements are modest (1–2%), these small gains are significant because they come with additional benefits:\n\n- **Enhanced Interpretability:** The clone-and-clamp mechanism produces a more interpretable latent graph by clearly separating known from uncertain edges.\n- **Stable Gradient Flow:** Our design avoids in-place modifications and contradictory KL signals, leading to more stable and robust training.\n- **Adaptive Behavior:** SGSI gracefully reverts to a standard VAE-based approach (e.g., NRI) when no prior knowledge is available, ensuring that even minimal domain guidance is exploited without harming performance.\n\nThus, even modest gains are valuable in high-performing scenarios and are complemented by improved interpretability and training stability, factors that are crucial for practical applications. We believe these advantages underscore the practical significance of SGSI beyond mere numerical improvements.\n\n**2. Scalability to Larger Real-World Graphs**\n\nWe agree that demonstrating scalability is important. However, obtaining a reliable dynamical system dataset with thousands of nodes and a trusted underlying structure remains challenging in our field. To address this, we conducted simulation-based experiments on a synthetic dataset with 5k nodes using subgraph sampling (akin to GraphSAGE). In this experiment, SGSI was trained with mini-batching, computing gating parameters only on sampled local neighborhoods. Our preliminary results, summarized in the table below, show that SGSI remains effective at scale, converging in under 60 hours while achieving meaningful AUROC improvements:\n\n|      | K.P. 10% | K.P. 20% | K.P. 30% | K.A. 10% | K.A. 20% | K.A. 30% | w. Spar. | w. In-deg. | w. Out-deg. | w. Both-deg. |\n| ---- | -------- | -------- | -------- | -------- | -------- | -------- | -------- | ---------- | ----------- | ------------ |\n| SGSI | 0.56     | 2.40     | 4.93     | 0.77     | 3.08     | 5.73     | 7.46     | 6.90       | 7.13        | 7.68         |\n\nThese findings suggest that the core mechanisms of SGSI (soft gating, cloning, and KL skipping) remain robust even for large graphs. \n\n**3. Computational Efficiency and Training Time**\n\nIn response to your request for explicit runtime data, we provide the following training-time comparison across our main datasets (Springs, NetSims, LI, LL, and the 100-node VN datasets):\n\n|       | Springs | NetSims | LI   | LL   | VN\\_SP\\_100 | VN\\_NS\\_100 |\n| ----- | ------- | ------- | ---- | ---- | ----------- | ----------- |\n| NRI   | 20.1    | 16.0    | 14.3 | 18.2 | 49.0        | 47.2        |\n| iSIDG | 42.2    | 36.9    | 48.1 | 50.6 | 100.6       | 97.8        |\n| SGSI  | 20.4    | 15.6    | 14.7 | 18.1 | 49.2        | 47.1        |\n\nSGSI consistently matches NRI’s training time (e.g., 20.4h vs. 20.1h on Spring Simulations and 47.1h vs. 47.2h for VN_NS_100), while methods such as iSIDG require ~1.5–2× longer. Yet recall that SGSI achieves 79.7% AUROC on Springs, which is higher than NRI. Overall, these results confirm that SGSI’s partial-knowledge gating imposes minimal additional cost while providing meaningful accuracy gains in structural inference.\n\nThank you again for your helpful comments, and we hope these newly added experiments bolster confidence in SGSI’s robustness, scalability, and efficiency.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "The methods and evaluation criteria are appropriate for the problem setting. The SGSI model is rigorously compared to multiple baselines, including NRI, MPM, ACD, iSIDG, RCSI, ALaSI, and SICSM. The evaluation uses standard metrics such as AUROC to measure structural inference quality. The paper also investigates different amounts of prior knowledge and studies the effect of global sparsity and node-degree constraints. The benchmarks selected, including NetSim, Spring Simulations, and StructInfer datasets, are relevant to relational inference tasks.", "theoretical_claims": "The paper presents a theoretical analysis of SGSI’s soft gating approach and its connection to the information bottleneck principle. The KL term is adapted to exclude known edges, and the latent adjacency matrix is factorized into deterministic and uncertain components. The derivations appear correct and align well with established VAE principles. The simplification of KL divergence, ensuring no contradictory prior signals, is a valid and well-motivated approach.", "experimental_designs_or_analyses": "The experimental design is sound, with multiple datasets and extensive ablation studies. The choice of datasets captures a broad range of real-world applications. The comparison with state-of-the-art methods is thorough, including controlled experiments that vary the proportion of known-present and known-absent edges. The loss curve analysis in ablation studies further supports the necessity of key SGSI components. The paper also accounts for hyperparameter tuning and provides practical considerations for sparsity and node-degree constraints.", "relation_to_broader_scientific_literature": "I am not familiar with this topic.", "essential_references_not_discussed": "na", "other_strengths_and_weaknesses": "Weaknesses:\n\nWhile SGSI performs well, some datasets show only marginal improvements over existing baselines.\n\nThe paper could further explore the applicability of SGSI to large-scale real-world graphs.\n\nDiscussion on computational efficiency is limited; reporting training time across datasets would be useful.", "other_comments_or_suggestions": "see weakness", "questions_for_authors": "see weakness", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GLmtHyBo44", "reviewer": "Reviewer_mhQC", "summary": "This paper proposes Soft-Gated Structural Inference (SGSI), a framework to solve the task of infer latent relational structures where additional prior knowledge can be incorporated. Theoretical analysis and experiments verify the effectiveness of the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your detailed review and the constructive feedback regarding SGSI. We address your concerns below (tables can be found at https://anonymous.4open.science/r/SGSI-Rebuttal-1614/Tables.pdf ):\n\n**1. Importance of Prior Knowledge**\n\nOur experiments across multiple benchmarks, including NetSim, Spring Simulations, and StructInfer, demonstrate that even a small fraction of reliable prior knowledge (e.g., 20% known-present or known-absent edges) can improve latent structure recovery by up to 9% AUROC in LL and several VN\\_NS datasets. We acknowledge that when available domain knowledge is sparse or less reliable, the improvements are naturally smaller. However, in many real-world scenarios (e.g., transportation networks, medical diagnostics), partial prior knowledge is both available and critical for ensuring interpretability and reliability. Importantly, SGSI is designed to gracefully revert to a standard VAE-based method when no prior is available, so its performance scales with the quality of the available knowledge.\n\n**2. Contribution of SGSI**\n\nWe respectfully disagree that our contributions are merely incremental. Our method introduces a novel soft gating mechanism that:\n\n- \\textbf{Clones and clamps} gating parameters to integrate known edges without causing in-place gradient conflicts.\n- \\textbf{Skips KL costs} for fully known edges, effectively reallocating “bits” to uncertain connections.\n- Integrates domain-specific constraints (global sparsity and node degrees) via soft penalties.\n\nThese design choices overcome critical challenges in merging data-driven latent inference with external knowledge, a capability absent in prior work. For instance, in gene regulatory network (GRN) inference, extensive experimental efforts identify a subset of true regulatory interactions. We evaluated SGSI on two GRN datasets, ventral spinal cord (VSC) development and gonadal sex determination (GSD) (Pratapa et al., 2020), using varying proportions of known-present edges. Table 5 in the attached link summarizes the results. The results show that incorporating prior knowledge not only improves AUROC but also helps the model produce more true positive edges, capabilities that are missing in prior approaches. We further foresee SGSI’s potential in fields such as physics, chemistry, and finance, although resource limitations preclude experiments on those domains at present.\n\n**3. Hyperparameter Sensitivity**\n\nSGSI introduces additional hyperparameters (e.g., $\\beta, \\lambda_{\\mathrm{sparsity}}$, and $\\lambda_{\\mathrm{deg}}$). In experiments, we selected these values using preliminary grid searches and cross-validation on a validation set. Our recommended ranges ($\\beta=1.0$ and $\\lambda_{\\mathrm{sparsity}}, \\lambda_{\\mathrm{deg}} \\in [10^{-4},10^{-2}]$) are consistent with common practices in VAE-based models. We plan to include additional guidance in the Appendix:\n\n- For very sparse graphs (e.g., local interactions in physical systems), higher $\\lambda_{\\mathrm{sparsity}}$ values are tested.\n- When node-degree constraints are precise (e.g., “exactly 2 neighbors”), we set a higher $\\lambda_{\\mathrm{deg}}$.\n- If the prior knowledge is approximate, lower penalty weights prevent overshooting data evidence.\n\nThese ranges avoid abrupt changes in adjacency, ensuring that SGSI functions as a softly guided VAE rather than a rigidly constrained model.\n\n**4. Theoretical Analysis and VIB Connection**\n\nOur VIB-inspired design, where known edges are excluded from the KL term, frees the model to devote its “bits” to uncertain edges, leading to notably better performance. Our ablation studies show that not skipping the KL for pinned edges degrades performance, confirming that when the model wastes capacity on fully known edges, it achieves lower accuracy. This aligns with the broader rationale behind VAE-based structural inference (e.g., ACD, iSIDG, RCSI), in which the Variational Information Bottleneck concept helps explain why prioritizing uncertain edges yields more effective adjacency discovery. While we do not claim a formal VIB theorem, we include this perspective because it clarifies the practical benefit of our skip-KL approach and highlights its theoretical consistency with known VAE methodologies.\n\n**5. Typos**\n\nWe appreciate your careful reading. All typographical errors have been corrected and will appear in the camera-ready revision.\n\n**6. Datasets**\n\nIn addition to the synthetic and benchmark datasets discussed in the main text, we evaluated SGSI on the PEMS (California Caltrans Performance Measurement System) dataset (see Appendix E.2). The PEMS evaluation, conducted on real-world sensor data, further demonstrates SGSI’s practical applicability and robustness in realistic settings.\n\nWe believe that SGSI’s novel soft gating mechanism significantly enhances structural inference by effectively leveraging prior knowledge while maintaining computational efficiency and scalability. Thank you for your valuable feedback.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "The proposed methods make sense.\n\nThe evaluated datasets contain simulated / synthetic datasets and benchmark datasets. However, it seems there is no clear scenario where the prior knowledge is crucial. It makes the contributions incremental.", "theoretical_claims": "There are no formal theoretical claims or proofs.", "experimental_designs_or_analyses": "The experimental designs and analyses are reasonable.", "relation_to_broader_scientific_literature": "It may influence fields like social network analysis or medical diagnostics where some prior knowledge is provided.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "- There are multiple hyperparameters in the method, as shown in Sec. 4.3. It is unclear how these are selected in experiments, and it seems they are sensitive to concrete scenarios.\n- The theoretical analysis is designed to show \"how deterministic and stochastic edges optimize the compression-prediction trade-off\". However, the connection to VIB does not present any meaningful conslusion and seems far-fetched.", "other_comments_or_suggestions": "-  ”rediscover” \n- ”bits”\n- Equ. 15, \".\" should be \",\"", "questions_for_authors": "See above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IoPEbYN0P8", "reviewer": "Reviewer_MPiQ", "summary": "The paper introduces a more controlled method for structural inference that allow us to imposed a series of constrained to the predicted structure. Specifically, it enables conditioning on a set of edges that must exist in the structure, enforcing the absence of certain edges, and controlling sparsity in terms of both the total number of edges and the degree of individual nodes.\n\nAt a technical level, the method it is a variation of the NRI paper with the following modifications: for restricting the edges, the method introduces a learnable gate per edge, which is \"clone-and-clamp\" for the edges that are in one of the 2 special sets; for the sparsity in terms of number of edges or degree of the nodes additional regularisation terms are added to the loss function.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your detailed review and constructive feedback. Below, we address your concerns (detailed tables can be found at https://anonymous.4open.science/r/SGSI-Rebuttal-1614/Tables.pdf ):\n\n**1. Choice of Evaluation Metrics and Downstream Tasks:**\n\nWe selected AUROC as our primary metric because it is standard in the structural inference literature, used by ACD, iSIDG, ALaSI, and RCSI, and effectively measures true positive rates over various thresholds, as demonstrated in Pratapa et al. (2020). In our context, where many methods output probabilities rather than hard binary edges, AUROC provides a comprehensive assessment of latent graph recovery.\n\nIn addition, although our primary goal is to infer the underlying interacting structure of dynamical systems, SGSI is also applicable to time-series forecasting. For instance, we report Mean Squared Error (MSE) for predicting 10 future steps on several datasets. Table 6 in the attached link compare NRI, SGSI without prior knowledge (SGSI raw), and SGSI with 20% known present edges.\n\nThese results show that while SGSI raw matches NRI, the integration of prior knowledge significantly improves forecasting accuracy. This dual capability demonstrates that our method is not only effective for structural inference but also beneficial for downstream prediction tasks.\n\n**2. NRI Variants**\n\nActually, we use both variants of NRI (with or without sparsity regularization), but figured out the the one with sparsity regularization outperformed the other variant by 1%-3% AUROC. So we refer to the one with sparsity regularization. We revised our paper to state it clearer.\n\n**3. Technical Contribution**\n\nModern autograd frameworks track tensor “versions” to compute gradients accurately. In-place masking, forcing gating values to 0 or 1 within the same tensor, disrupts this mechanism, causing version mismatches and unstable gradients. Our clone-and-clamp strategy first duplicates the gating vector, then clamps the copy, thereby preserving the original tensor’s autograd history. This ensures that (1) the original logits $\\theta_e$ remain intact for backpropagation, enabling stable gradient flow, and (2) only the cloned, clamped tensor is used in message passing.\n\nOur ablation experiments (Section 5.3) clearly show that omitting cloning leads to sharp gradient spikes and oscillatory training, while not skipping the KL for pinned edges results in contradictory updates that inflate the loss and harm AUROC. These findings confirm that simply masking in-place degrades gradient quality, whereas our clone-and-clamp technique maintains stable and consistent gradient propagation.\n\n**4. Limitations of the Learnable Gate**\n\nOur soft-gating mechanism enables the use of roughly estimated prior knowledge, such as overall sparsity or node degree constraints, that is often stable and transferable within our domain. In many application areas (e.g., transportation networks, gene regulatory networks), the underlying graph structures exhibit similar patterns. Thus, the partial knowledge (e.g., a typical density or expected node degree) is largely sharable across datasets. Moreover, our design allows the gating penalties to be adjusted; for instance, one can fine-tune the model through a pretraining and fine-tuning process if needed.\n\nAs detailed in our response to Reviewer vBqs, SGSI is robust against moderate errors or deviations in prior knowledge. Although the penalties (e.g., the sum of gating values for sparsity or node-level degree sums) might appear straightforward, their true strength lies in their seamless integration with our variational framework:\n\n1. They are **soft**: the model can override them when data strongly suggests an alternative structure.\n2. They are **partial**: penalties can be applied selectively, only to nodes or edges where reliable prior knowledge exists.\n3. They **co-exist** with our skip-KL mechanism for pinned edges, ensuring that enforcing known edges does not conflict with the overall latent representation.\n\nThis flexibility not only facilitates the transfer of prior knowledge across related domains but also makes SGSI adaptable to different datasets within the same field. We note that approaches like META-NRI have similarly emphasized leveraging common structural priors across domains, further supporting our design choices.\n\nIn summary, our soft gating penalties act as additional, adjustable levers that exploit stable, transferable domain knowledge, ensuring that SGSI remains effective whether prior knowledge is abundant or only approximately known. We believe this design is both practical and powerful, and we are confident that these clarifications strengthen the final manuscript.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "- In all the experiments, the paper reports only AUROC for the predicted structure. Is there a specific reason why AUROC was chosen to assess the quality of the prediction? How would the models compare using more discrete metrics, such as structure accuracy?\n\n- The main focus of the method is to predict a constrained latent structure. However, without a downstream task, the practical applicability of this approach remains unclear. Similar to previous works, such as NRI and ACD, it is important to quantify how well a decoder utilizing the predicted latent structure performs on a downstream task. While I understand that achieving a worst performance than the baselines in terms of downstream tasks may be expected (given that the model has the additional advantage of enforcing hard constraints that could be crucial) it is still essential to demonstrate that the predicted structure remains useful to some extent. These evaluations would provide a more comprehensive understanding of the model.", "theoretical_claims": "The theoretical claims are correct.", "experimental_designs_or_analyses": "- While less explicit, the NRI paper also introduce a sparsity regularisation in the form of a modified prior distribution, which assigns a higher likelihood to non-edges compared to edges. Does the baseline experiments presented in the paper control that sparsity in any way?", "relation_to_broader_scientific_literature": "The paper does a good job on summarizing the existing literature in relational inference.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "- In my opinion the technical contribution of the paper is limited. The method differs from previous work in three main aspects: the clone-and-clamp technique and two regularisation losses. The regularisation losses are trivial and requires strong domain knowledge. Without such prior knowledge, the sparsity and degree became just another hyperparameters. Moreover it is unclear how imposing them would affect a potential downstream task.  Regarding the clone-and-clamp technique, it is still unclear to me why the quality and stability of the gradients are superior in this case compared to simply masking the edges, as done in the baselines. I acknowledge the advantages brought by eliminating those terms from the KL divergence loss, however I still have concerns about how the technique improve the gradient quality. Please provide more details or additional experiments to clarify this point.\n\n- The gating mechanism relies on a learnable set of parameters, one for each edge. This impose a strong limitation into the model, since it implies that the structure behaves similarly between different examples in the dataset. In addition to that, it makes it impossible to generalise to examples with a different number of nodes.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Please see the comments above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents Soft-Gated Structural Inference (SGSI), a framework for incorporating partial prior knowledge into VAE-based structural inference. The proposed method uses a combination of soft gating, clone-and-clamp mechanisms, and adaptive regularization to handle known edge constraints, sparsity, and degree priors in a stable and interpretable way.

The reviewers generally appreciated the novelty and empirical strength of the method. Multiple reviewers praised the paper for integrating domain knowledge into structural inference without destabilizing learning, noting that SGSI achieves notable gains in AUROC and provides interpretability and robustness. Concerns were raised regarding the practical availability of prior knowledge, the technical novelty of certain components (e.g., clone-and-clamp), and the lack of downstream task evaluation, which the authors addressed with new experiments and real-world data. These clarifications were well-received by the reviewers. 

Overall, this submission is well-positioned for acceptance, as it provides a practical, extensible solution to an important problem in structure learning, with strong experimental validation and thoughtful design choices. Therefore, I recommend acceptance.
Decision: Accept (poster)