InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
Authors: Tianyu Cui
Song-Jun Xu
Artem Moskalev
Shuwei Li
Tommaso Mansi
Mangal Prakash
Rui Liao
Venue: ICML 2025 poster (2025)
Keywords: Gene Regulatory Networks inference
Informative priors
scRNA-seq
Variational Autoencoder
Primary area: applications->health_medicine
TL;DR: InfoSEM is a VAE-based generative model for GRN inference that incorporates informative priors on the adjacency matrix.

Abstract: Inferring Gene Regulatory Networks (GRNs) from gene expression data is crucial for understanding biological processes. While supervised models are reported to achieve high performance for this task, they rely on costly ground truth (GT) labels and risk learning gene-specific biases—such as class imbalances of GT interactions—rather than true regulatory mechanisms. To address these issues, we introduce InfoSEM, an unsupervised generative model that leverages textual gene embeddings as informative priors, improving GRN inference without GT labels. InfoSEM can also integrate GT labels as an additional prior when available, avoiding biases and further enhancing performance. Additionally, we propose a biologically motivated benchmarking framework that better reflects real-world applications such as biomarker discovery and reveals learned biases of existing supervised methods. InfoSEM outperforms existing models by 38.5% across four datasets using textual embeddings prior and further boosts performance by 11.1% when integrating labeled data as priors.

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
Tianyu Cui
Song-Jun Xu
Artem Moskalev Shuwei Li
Tommaso Mansi
Mangal Prakash
1* Rui Liao
Abstract
Single-cell RNA sequeneing (scRNA-seq) has revolution- ized GRN inference by enabling high-resolution profiling of cell-specific regulatory interactions. However, scRNA- seq data are noisy, sparse, and high-dimensional (Pratapa et al., 2020; Wagner et al., 2016; Dai et al., 2024), requir- ing advanced computational approaches. Methods have evolved from co-expression frameworks (Chan et al., 2017; Kim, 2015) to cutting-edge machine learning (ML) and deep learning (DL) models (Yuan & Bar-Joseph, 2019; Wang et al., 2024; Anonymous, 2024; Shu et al., 2021), with fur- ther improvements leveraging complementary but hard to obatin perturbation experiments, RNA velocity, and chro- matin accessibility inputs (Chevalley et al.. 2022; Atanack- ovic al., 2024; Yuan & Duren, 2024).
Inferring Gene Regulatory Networks (GRNs) from gene expression data crucial for under- standing biological processes. While supervised models are reported achieve high performance for this task, they rely costly ground truth (GT) labels and risk learning gene-specific bi- ases-such as class imbalances of GT interac- tions-rather than true regulatory mechanisms. To address these issues, we introduce InfoSEM, an unsupervised generative model that leverages textual gene embeddings informative priors, improving GRN inference without GT labels. In- foSEM can also integrate GT labels as an addi- tional prior when available, avoiding biases and further enhancing performance. Additionally, we propose a biologically motivated benchmarking framework that better reflects real-world appli- cations such as biomarker discovery and reveals learned biases of existing supervised methods. In- foSEM outperforms existing models by 38.5% across four datasets using textual embeddings prior and further boosts performance by 11.1% when integrating labeled data as priors.
GRN inference methods can be broadly classified su- pervised or unsupervised. Supervised models use ex- perimentally derived ground truth (GT) labels, such as ChiP-seq data, where known TF-TG interactions guide learning. While they achieve high performance (AUPRC ~0.55) (Chen & Liu, 2022: Wang et al., 2023), their re- liance on costly GT labels limits applicability. Unsupervised methods which infer regulatory relationships directly from gene expression data without using any known interactions, present an attractive alternative but typically lag in perfor- mance compared to supervised models (Chen & Liu, 2022; Wang et al., 2023; Mao et al., 2023). Bridging this gap requires advancing unsupervised GRN inference methods.
1. Introduction
Gene Regulatory Networks (GRNs) govern cellular pro- cesses by capturing regulatory relationships essential for gene expression, differentiation, and identity (Karlebach & Shamir, 2008). Represented as directed graphs, they typ- ically feature transcription factors (TFs) regulating target genes (TGs), with non-coding RNAs also playing key roles. GRNs have diverse applications, including mapping molec- ular interactions (Basso et al., 2005), identifying biomark- ers (Dehmer et al., 2013), and advancing drug design (Ghosh & Basu, 2012).
To address this, introduce InfoSEM, unsupervised generative framework trained with variational Bayes that leverages textual gene embeddings as informative priors. By integrating prior biological knowledge, InfoSEM sub- stantially improves GRN inference over existing models. Moreover, InfoSEM can incorporate GT labels as an ad- ditional prior when available, rather than using them for direct supervision, avoiding dataset (gene-specific) biases and improving performance further.
Beyond model development, reliably evaluating GRN in- ference methods is equally important. Existing supervised learning based GRN inference benchmarks typically as- sume all genes are represented both the training and test sets, and only the regulatory links in the test set differ from those seen during training (unseen interactions between
'Shared last authors 'Johnson and Johnson Innovative Medicine. Correspondence to: Tianyu Cui <tcui8 @ its jnj.jnj.oom.
Proceedings of the ndnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
seen genes, Section 4). While suitable for predicting inter- actions among well-characterizee genes, this setup does not reflect many real-world applications such as biomarker dis- covery and rare cell-type studies (Lotfi Shahreza et al., 2018; Ahmed et al., 2020), which often involve genes whose inter- actions are completely absent from the training data (inter- actions belween unseen genes in Section 4), thus requiring models to generalize beyond the set of genes encountered during training.
Liu, 2022; Chen & Zou, 2024; Haury et al., 2012; Moerman et al., 2019). The scRNA-seq gene expression data, com- prising P genes and N cells, is represented as X E IR*,,, where 2ij E IR denotes the expression of gene in cell j. and at; is the i-th row of X, corresponding to the expression profile of v; across all cells.
2.1. Supervised GRN Inference
When partial information about the adjacency matrix Y is available from sources like ChIP-seq or databases such as the Gene Transcription Regulation Database (Yevshin et al., 2019), GRN inference can be framed as supervised learn- ing (SL.) task, where the model is trained on labeled data with known interactions in Y serving as labels. The model learns to predict the probability of an interaction Yiu being true based on observed data by minimizing a cross-entropy loss, i.e., assuming a Bernoulli likelihood. Specifically, SL methods aggregate representations of genes and k. denoted as s; and sy. which could be function of their expression profile æ, and a and predict the probabílity of interaction Yik using function fa on the aggregated representation:
To bridge this gap, we propose biologically motivated benchmarking framework evaluating interactions between unseen genes, better aligning with many real-world applica- tions. This shift in evaluation perspective also inadvertently highlights that, in prior benchmarks, supervised models may have relied on gene-specific biases of the dataset, e.g., class imbalance in known interactions for each gene. This under- scores the need for careful evaluation to distinguish between performance gains driven by genuine biological insights from scRNA-seq and those influenced by dataset biases.
In summary, our work makes the following contributions:
We present InfoSEM, unsupervised generative frame- work that leverages textual gene embeddings as priors and can seamlessly integrate GT labels as an additional prior (when available), avoiding dataset biases and improving GRN inference.
Pik = fagg( s,, sk)), ysk ~ Bernoull(pp...
(I)
Deep learning methods such as CNNC (Yuan & Bar- Joseph, 2019), DeepDRIM (Chen et al., 2021), and GENELink (Chen & Liu, 2022) leverage CNNs, GNNs, or transformers to aggregate pairwise gene expressions or their deep representations for interaction prediction (Wang et al., 2023; Mao et al., 2023; Xu et al., 2023). The SL frame- work can also flexibly incorporate pre-trained embeddings, such as BioBERT embeddings in seGREAT (Wang et al., 2024) or ScBERT embeddings combined with GENELink representations in scTransNet (Kommu et al., 2024).
We reveal limitations in existing supervised learning based GRN inference benchmarks, showing that supervised models may exploit dataset biases, such as class imbal- ance of each gene, rather than capturing true biological mechanisms from scRNA-seq.
We propose new evaluation framework focused on regu- latory interactions between unseen genes, better aligning with world applications such as biomarker discovery.
2.2. Unsupervised GRN Inference
Unsupervised learning (USL) methods infer gene relation- ships without labeled data, relying solely on gene expression data and more closely mirroring real-world scenarios (Prat- apa et al., 2020) where labeled data is unavailable. Ap- proaches include information mncory-based methods (e.g., partial Pearson correlation (Kim, 2015). mutual informa- tion (Margolin et al., 2006), and PIDC (Chan et al., 2017)) and self-regression models, which predict a gene's expres- sion based on the expressions of all other genes.
Finally, we demonstrate that our InfoSEM improves GRN inference by 38.5% over models without priors and achieves state-of-the-an art performance, even surpassing su- pervised models. Integrating GT labels as additional prior further boosts performance by 11.1% while mitigat- ing dataset biases.
2. GRN inference problem
Let G = (V,Y) represent a GRN, where V = (vi....,up denotes the set of P genes (nodes), including transcription factors (TFs) and target genes (TGs). The adjacency ma- trix Y E 10. ) encodes regulatory interactions, where = 1 indicates that TF es regulates TG Uj. We assume access to scRNA-seq gene expression data and the goal is to infer the adjacency matrix Y, a setup commonly used in recent studies (Shu et al., 2021; Anonymous, 2024; Chen &
........0000
2; (() 32
xp) + Zi,
(2)
where 2; is Gaussian noise term. The feature importance of Φt in fi indicates the interaction effect from gene k to i. TIGRESS (Haury et al., 2012) solves a linear regression for each f, in Eq.2.
X A Z.
(3)
2

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
A'
H
A
Z
A
Z
A
A
Z
W
0
O
(a) DecpSEM Gesesttive Moded
(b) DeepSEI Modd
(c) InfoSEM-B Genssttive Madid
(d) InfoSEM-BC Genazative Mode
Figure 1. (a) Generative model of DeepSEM: (b) Inference model of DeepSEM: BioBERT eene-embediing priors H on interaction effects; (d) Generative model of InfoSEM-BC: InfoSEM with both BioBERT gene-embedding priors on interaction effects A" and known interactions (e.g-ChlP-sse)) priors on logit of the probability of interactions A'. Solid and dashed lines represent stochastic and deterministic relations respectively.
(c) Generative model of InfoSEM-B: InfoSEM with
where all diagonal elements in the weighted adjacency matrix A E RPxx are and an L1 regularization is ap- plied on A. A post-hoe threshold, e-g., only keeping the top 1% interactions, can be further applied on weighted adjacency matrix A to obtain adjacency matrix Y. GF- NIE3 (Huynh-Thu et al., 2010) and GRNBoost2 (Moerman et al., 2019) use random forests and gradientboosting to enhance model flexibility and feature importance represen- tation. Equation 3 resembles the linear structural equation model (SEM) in Bayesian networks (Zheng et al., 2018), with added acyclicity constraint on A. However, classi- cal GRNBoost2 still outperforms even the latest Directed Acyclic Graphs (DAG) learning algorithms due to feed- back effects (Chevalley et al., 2022). DeepSEM (Shu et al., 2021) extends Eq.3 by modeling gene expression in latent space without the acyclicity constraint, using 8- - autoencoder (VAE) denoise data, and outperforming tra- ditional USL methods (Shu et al., 2021; Zhu & Slonim, 2024; Pratapa et al., 2020). However, it lacks support for incorporating external priors, which have shown to provide accuracy improvements for recent SI. methods ( (Anonymous, 2024; Wang et al., 2024; Kommu et al., 2024).
specified by a weighted adjacency matrix A by first generat- ing random noise Z, and then solving X (1 --) - (Yu al., 2019). Similarly, DeepSEM defines generative model, shown in Figure I (a), as follows:
A ~ p A)) Z p(Z),
(4)
2= (1 A )-) 2, X ~ ( (X 2),
where the GRN structure, represented by A, is modeled on the latent space 2 of X, rather than directly on X. Both and 2 share the same GRN structure. We use X ~ pe ( A) to simplify the second line the generative model in Eq.4. All distributions in Eq.4 are fully factorized:
((A) II 6,k
Laplace(an:) 0,.))
2(Z) = II
N (zyi 0,o?),
(5)
(x 2)
II 1.j
( (o( go(2i)))
where as represents the interaction effect from gene i to gene h and ziy is the corresponding latent variable of Zij. i.c., the expression of gene i in cell j. A Laplace prior is given to A to encourage sparsity where the hyper- parameter da controls the sparsity level. Similar to the vanilla VAE (Kingma, 2013), a zero-mean Gaussian prior with standard deviation a₂ is applied to the latent variable Z and both the mean and variance of the Gaussian like- lihood of X are given by the generation networks fo(.) and go(-) parametrized by 0. To learn DeepSEM, an inference network that approximates the posterior of Z, Z go (X),, A), is introduced as follows:
3. InfoSEM: Informative priors for DeepSEM
Inspired by DeepSEM, we carefully design InfoSEM, novel and principled generative model that incorporates multimodal informative priors: textual gene embeddings from pre-trained foundation models and known regulatory interactions (when available). InfoSEM is designed for sce- narios both with and without labeled interactions, addressing limitations in the original DeepSEM framework. We begin by reviewing DeepSEM and then detail our approach to integrating these priors.
2 ~~ (0( Z = (I AT)2,
(6)
3.1. Introduction to DeepSEM
where qs (2)x) is also fully factorized:
The linear SEM model in Eq. 3 can be viewed genera- tive model: it generates the dataset X with a GRN structure
9o (2(x)) [((.)
(7)
3

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
A is learned using its maximum a posteriori (MAP) estimate A, which is equivalent to using Dirac measure the approximated posterior distribution, g (A) = 6(A A), in the variational Bayes framework. The whole inference model is shown in Figure (b). Therefore, Á, 0, & are optimized by maximizing a lower-bound of the likelihood, i.e. evidence lower-bound (ELBO):
where [h., h&] concatenates the gene embedding of genes and l into 2d-dim row vector and RRx2dd is the parameter to learn using a MAP estimate with a prior p(w) = (0, The prior mean in Eq.9 can be in- terpreted as a linear model built on the textual gene em- beddings for predicting their interactions. Although more flexible nonlinear models, e.g., MLP, can be applied, lin- ear models on pre-trained embeddings have already been successful in several gene-level tasks (Chen & Zou, 2024).
Ix))
=log| / po X 22 A)P(A)((2 bo(z|x,A)dZdl ())
The prior distribution defined in Eq.9 makes similar genes have similar interaction effects with other genes. Intuitively, if gene 2 and gene 3 are functionally close, i.e., h2 x h3, the prior defines a similar high- Sigh-density region for 012 and 413 as they have similar prior mean: [h1, h2] jia Rs [h,, ha ww"""" Moreoever, the prior is asymmetrie, p(a,s) (w) * p(axs/w). which reflects the asymmetric nature of GRN.
2E (
log Po (0)2, A) + (ogg(()
(8)
-KL (2x A)b(()
O))
From here on, we refer to this model, which uses BioBERT embeddings as priors, as InfoSEM-B.
where the first term the expected reconstruction error of the gene expression matrix X, ((A)(A) regularizes the is the Kullback-Leibler divergence that regularizes the approx- imated posterior of 2 and is weighted by 3 in practice.
MAP estimate of A. and KL A))
3.3. Incorporate both gene embeddings from pretrained language models and known gene-gene interactions
Although not always readily available. prior knowledge of gene-gene interactions can be obtained in specific scenar- ios, such as those studied in (Yuan & Bar-Joseph, 2019; Anonymous, 2024; Chen & Liu, 2022), where ChiP-seq experiments or similar methodologies provide ground truth interaction data for subsets of Y. When available, these partially observed interactions offer valuable biological in- sights that can guide GRN inference. Here, we show how to incorporate these known interactions in addition to the textual gene embeddings previously described.
In following subsections, we introduce our model InfoSEM by proposing flexible and informative priors for p( A) to re- place the weakly informative Laplace prior in 5q.5. Specif- ically, InfoSEM models the interaction effects, guided by textual gene embeddings, and the probability of interaction, informed by known interactions, when available.
3.2. Incorporate gene embeddings from pretrained language models
Given the frequent unavailability of costly experimental readouts, such as chromatin accessibility, RNA velocity, perturbation experiments, and accurate pseudo-time annota- tions, we propose the use of readily available priors. One such prior is gene embeddings derived from textual gene descriptions, which have been successfully utilized in recent supervised learning frameworks ( (Wang et al., 2024; Kommu et al., 2024). Gene-level information can assist GRN infer- ence: if gene i and gene l are functionally similar, they may also have similar interaction effects with other genes. We use the d dimensional embedding h; Ix of the textual description of the gene name from BioBERT (Lee et al., 2020), a language model pretrained on extensive biomedical literature, to represent the function of gene i. We design a prior of interaction effect informed by language model embeddings as shown in Figure 1 (c):
Incorporating known gene-gene interactions comes with a natural challenge in that the partially observed Y is binary which does not inform the continuous weighted adjacency matrix A directly, but it can inform the probability of inter- actions. Therefore, we propose to decompose the weighted adjacency matrix A into A E RxP, representing the mag- nitude of the interaction effect, and A' E RxP,, with each element a/ representing the logit of the probability that gene interacts with gene k:
A = A OO(A),
(10)
where af-) is the sigmoid function and @ is element-wise product (Hadamard product) between them, as shown in Figure I (d). We work on the logit space of probability to remove the necessary constraints during the model training. The prior of A is informed by the gene embeddings from pretrained language models in the same way as Eq.9:
P(A w) II p(ask hh, he, w) 1.k
Pe (A w) = II
Laplace(afa [h;, hu|w" .a). (11) i,k
(9)
Laplace(ax: (ha, hg/wo" Ja):
We define a prior on A' using the partially observed as
4

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
following:
(Y II pi(alubuus) x,K
(12)
TO
TO
where
(a) tnin ort (kmmen fortnining)
(b) somgrme
if yik 1, (00)..(0.05))..))) if You O, Po(d =
N ((lo
Figure 2. Strategies of train-test splits of the ground-truth network. containing three transcription factors (TFI-TF3) and six target genes (TGI-TG6), when it is used during training. Solid and dashed lines represent positive and negative interactions. (a) Blue links (-+) represent known interactions used in the training set. (b) Green links (-+ represent the test set D i.c., pairs of genes for which the individual genes are seen in train data, but not their interaction. (c) Red links (-+) represent the test set D i.e., pairs that both genes and their interactions are unseen in the train data.
(13)
000,,
unknown,
Intuitively, 13 sets the mode of the prior probability that gene interacts with gene to 0.95 if we know they interact and 0.05 if they do not. The hyper-parameee precision of the labels. We truncate the binary label to 0.95 and 0.05 to ensure numerical stability (Skok Gibbs et al., 2024). If Yik: is not observed, we use a non-informative uniform distribution U [00,,000 oo] as the prior of the logit, rep- resenting that the mode of the prior probability is 0.5.
di controls the
4. A new benchmarking framework for evaluating GRNs
We will henceforth refer to this model, which incorporates BioBERT embeddings well as known interactions (from e.g., ChIP-see) when available as priors, InfoSEM-BC.
Evaluation methodologies for GRN inference models play critical role in validating their biological relevance and utility. Existing GRN evaluation benchmarks, while suit- able for specific tasks, have limitations. We highlight these shortcomings first and propose biologically motivated framework to address them.
3.4. Learn InfoSEM with variational Bayes
We train InfoSEM-B (Section 3.2) and InfoSEM-BC (Sec- tion 3.3) using variational inference.
We use MAP estimates to infer all variables related to the weighted adjacency matrix, i.c., w in both models, A in InfoSEM-B, and A and A' in InfoSEM-BC, given the data
4.1. Limitations of existing supervised learning benchmarks
X. We use a full-rank matrix A, A E RR PxP for A and A i.e., (A) 8(A- = A) and qá- (A*) = 5(A A²). We use a low-rank MAP estimate with rank < P to infer A', motivated by the fact that the maximum rank of the adjacency matrix of a GRN is the number of transcription factors, as it represents the interactions from transcription factors to target genes (Li et al., 2020; Weighill et al., 2021). Therefore, we use A.,..
When using parts of the ground-truth network train GRN inference model, careful train-test split strategies essential for reliable evaluation. In most supervised GRN inference studies (Chen & Liu, 2022: Wang et al., 2023; Mao et al., 2023; Xu et al., 2023; Wang et al., 2024; Kommu et al., 2024), datasets are split by gene pairs, where edges for each TF are randomly divided into training and test sets. This results in a test set, Diest containing unseen interactions between seen genes. For instance, in Figure 2 (b), the link from TF2 to TG1 is in Dieg training, though both TF2 and TG1 appear in the training set (Figure 2 (a)). This means that TFs and TGs in test sets are also in training sets, with only the regulatory interactions (edges) being distinct between the training and test sets. Furthermore, the class imbalance for interactions associated with each TF remains consistent across Dirain and Dust This approach has the following major limitations, which are further explored in Section 5:
(A') ) = 8(A' A,,4(), where A' &R A/, R. and A A/, has a rank h.
soon unseen during
We derive the ELBO of InfoSEM-B with above model to be
LIntosEMaB f, Ø, w)
logro( (X| A) =E (Z2((()
logr(A(H, (14)
+ +logPu KL @o (o. A)lp())
and the ELBO of InfoSEM-BC to be
CIGGSEM-BCAA A,, 1/, 0, o. w)
log (x A ((AAA)))
Risk of memorization due class imbalance: The par- tially observed ground-truth GRN networks are often heavily imbalanced (Anonymous, 2024; Pratapa et al., 2020): most TFs have far more negative interactions (no regulation) than positive ones (Figure (b) right). When all genes are represented in both training and test sets,
= - A/D
(15)
+ lo
pm (A) A(1) + (1))
- KT. (2 1 2 a 4}))p(())
We provide the detailed derivation the Appendix
5

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
Methods
ScRNA-scq
Known GRN
External pricr
Framework
setup followed by comparison of our proposed InfoSEM model with state-of-the-art supervised and unsupervised SL
One-hot LR
X
Matrix Completion
x
x
SL
ScGREAT
/
SL
models. For reproducibility, all experimental details are available in Appendix D.
GENELink
SL
GRNBoost2
8
USL
DeepSEM
1
x
x
USL.
InfoSEM-B (Ours)
USL
For all experiments, we use scRNA-seq datasets of four cell lines from the popular BEELINE suite (Pratapa et al., 2020), including human embryonic stem cells (hESC) (Yuan & Bar- Joseph, 2019), human mature hepatocytes (hHEP) (Camp et al., 2017), mouse dendritic cells (mDC) (Shalek et al., 2014), and mouse embryonic stem cells (mESC) (Hayashi et al., 2018). We consider two available ground-truth net- works, cell-type specific ChIP-seq, collected from databases such as ENCODE and ChlP-Atlas, on the same or similar cell type, and non-cell-type specific transcriptional regula- tory network from BEELINE (Pratapa et al., 2020).
InfoSEM-BC BC (Ours)
USI.
Table 1. Properties of a list of benchmarking methods. Existing methods that make use of known GRN interactions treat them as prediction targets in supervised learning (SL) framework while our approaches consider them as prior for constructing the scRNA- seq in unsupervised learning (USL) framework.
models can memorize gene-specific patterns (e.g.. gene IDss) by exploiting their node degrees from the GT net- work rather than learning true regulatory mechanisms using the gene expression data. Such memorization un- dermines the biological validity of the inferred networks.
We compare nine methods, including: 1. random baseline; 2. two trivial methods that do not use any information from scRNA-seq: one-hot LR (a logistic regression that takes the concatenation of the one-hot embedding of two genes to predict the ground-truth interactions) and matrix completion (MatComp) that imputes the partially observed adjacency matrix Y using low-rank decomposition (Troyanskaya et al., 2001); 3. two recent supervised methods with state-of-th the- art reported performances: SeGREAT (Wang et al., 2024) and GENELink (Chen & Liu, 2022); 4. two -tate-of-the-art unsupervised methods: GRNBoost2 (Moerman et al., 2019) and DeepSEM (Shu et al., 202 5. our proposed InfoSEM- B and InfoSEM-BC models. Inputs and training objectives for all models are summarized in Table 1.
Inadequate representation of real-world scenarios: In real- world cases such as biomarker expansion studies, the interaction (degree) information of most genes is not rep- resented in the partially observed GRN (Lotfi Shahreza al., 2018). For instance, TF3, TG3, and TG6 in Figure 2 (c) exhibit this scenario. Evaluating models on unseen genes (genes not present in the training set) reflects their ability to generalize to new biological contexts. This scenario is not captured by existing benchmarks.
4.2. Proposed benchmarking framework
We consider two types of evaluation metries: accuracy- based metric: AUPRC (Yang et al., 2022), and rank-based metric (Pratapa et al., 2020): number of positive interactions among the top 1% (Hit@1 1 1%) predicted interactions. We repeat the train-test splits 10 times with different random seeds to estimate the error bar for each method.
In this work, we construct another test set, Diest Pussenn con- taining interaction between unseen genes, i.e., all genes in Diest
unseen are not in the training set (see Figure 2 (c) for ample). The model performance and generalization ability on Diest unseen would reflect the level of biology that the model has learned from the gene expression data. In practice, we randomly divide all TFs and TGs with known interactions into four sets: seen and unseen TFs, e.g., [TF1, TF2] and [TF3] in Figure and seen and unseen TGs, e.g., [TGI, TG2, TG4, TG5] and [TG3, TG6), with ratio 3:1. All links between unseen TFs and unseen TGs are in Diest (e.g., TF3 - TG3). We then randomly divide all interac- tions between seen TFs and seen TGs into training Ditain and Diest with a ratio 3:1 using split strategies in existing works. Essentially, we leave some TFs and TGs out when constructing the training set whose interactions are then considered the unseen gene test sets Diest unseon
5.1. Exploring the biases with existing benchmarks
Under the existing benchmarks involving unseen interac- tions between seen genes, we observe that the performance of existing supervised and unsupervised methods aligns with prior studies (Wang et al., 2024; Chen & Liu, 2022; Pratapa et al., 2020). However, no previous work has compared these methods to simple supervised baselines, one-hot LR, which uses only one-hot ene-boteebeediingg of gene IDs as features, and a matrix completion baseline, which simply imputes the partially observed Y without any additional information. None of these use any information from scRNA-seg data, Surprisingly, both one-hot LR and matrix completion per- form similarly to state-of-the-ar supervised models such as scGREAT and GENELink across all cell lines (Figure (a) first column for hESC and hHEP cell lines, full results in Appendix F. for all cell lines show similar trend).
5. Experiments
We begin by experimentally validating the limitations of ex- isting GRN inference benchmarks mentioned in the previous subsection to motivate the need for our new benchmarking
6

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
genes
Randen
TRAP2A
SNA12
64
LA-
Spearman rho: 0.99 Histogram
One-hot LR
Heltggam KDt
TF Imbalance
0.0
0.2
0.5
1.0
0.4
100
SXGREAT
0.2
0.0
-5
10
Confficient
Frequency
0.5
Prediction
0.5
1.0
0.0
Prediction
D.5
1.0
(a) Performnce of existing methods
(b) One-hot LR coefficients
(c) One-hot LR and REGREAT predictions
Figure 3. (a) AUPRC with corresponding standard error of the mean of existing GRN inference models with cell-type specific target on both unseen interactions between seen genes (first column) and interactions between unseen genes (second column) test sets. One-hot LR and matrix completion without gene expression data achieve similar performance as several latest supervised methods (scGREAT. GENELink) using gene expression. (b) Relations between class imbalance level and one-hot LR coefficient of each TF. The x-axis shows the coefficient one-hot embedding in L.R. The y-axis represents the imbalance level (proportion of positive interactions) of each TF with histogram shown on the right. (c) Histograms and kernel density estimations of predicted probabilities from one-hot LR and scGREAT on interactions in Dies with two TFs: TFAP2A (left) and SNA12 (right), with training class imbalance level 0.89 and 0.06 respectively.
This performance is driven by one-hot LR's s ability to exploit the node degree from the partially known GRN. Specifically, the one-hot embeddings enable the model to memorize gene- specific biases, such as the probability of a transcription factor (TF) regulating target gene (TG). To validate this, we analyzed the correlation between the class imbalance level (the ratio of positive to negative interactions) of each TF and the corresponding one-hot coefficient in the one-hot LR. Figure 3 (b) shows a strong rank correlation, supporting that one-hot LR predicts interactions based on dataset bi- ases to match the class imbalance level of the corresponding TF, rather than biological relationships. Similarly, Mat- Comp works well by learning degree biases in the partially observed network of genes with low-rank latent features, which generalizes effectively to unseen interactions between those seen genes in the partially observed network.
using scRNA-seq data. In this scenario, we observe a signif- icant performance drop for supervised GRN inference meth- ods, averaging 42% for cell-specific ChiP-seq and 79% for non-cell-ppecffic ChiP-seq (see Figure 3 (a) second column, Table 2, and Appendix Table 6), even underperforming the random baseline (dashed green line in Figure 3 (a)). In con- trast, unsupervised methods, especially DeepSEM, outper- formed supervised methods on three datasets (hESC, hHEP, mDC) for cell-specific ChIP-seq without using known in- teractions. This further confirms the dependence of super- vised methods on dataset speciffc biases. Using simple techniques such as downsampling to address class imbal- ance in supervised methods reduces their accuracy inflation existing benchmarks but does not improve their accu- racy on unseen genes scenario (see Appendix E for details). This trend holds across both cell-type-specific and non-cell- type-specific benchmarks, and for metrics such as Hit@ 1% (additional results in the Appendix F).
In imbalanced datasets, where certain genes are predomi- nantly associated with positive or negative interactions, su- pervised models can achieve high performance by learning these biases, rather than meaningful biological relationships from gene expression data. The cross-entropy loss allows supervised models (Eq.1) to exploit this class imbalance by learning, for example, how likely a TF is to regulate TGs, or how likely a TG is to be regulated, without rely- ing on gene expression data. This shortcut allows models to predict unseen interactions between seen genes based solely on these biases. For illustration, both one-hot LR and scGREAT predict unseen interactions of a seen TF (e.g.. TFAP2A in Figure 3 (c)) positive if the TF has more positive interactions in the training set and vice versa (e.g.. SNAI2).
5.2. Utility of informative priors for InfoSEM
Next, we study the performance of our proposed InfoSEM models on our introduced unseen genes benchmark. Ta- ble and Appendix Table demonstrate that InfoSEM-BC and InfoSEM-B emerge as the best-performing models on all datasets that we tested for both cell-specific and non- cell-specific ChIP-seq. InfoSEM-B, which does not use known interactions, always improves upon DeepSEM on all datasets that we tested for both cell-specific and non-cell- specific cases by 25% and 52% on average, respectively. InfoSEM-B, -B, without using any known interaction labels, achieves top-2 performance for both AUPRC and Hit@ 1% metrics on 3 out of 4 datasets, even when compared to SL baselines that incorporate known interactions (see Table 2 and Appendix Tables 6 and 7).
To systematically confirm further whether sophisticated su- pervised models, such as scGREAT and GENELink, also rely on dataset biases, we evaluated them on our new bench- mark designed to predict interactions between unseen genes
7

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
BESC
НHEP
mDC
MESC
AUPRC
Hit 1%
AUPRC:
Hir@ 1%
AUPRC
Hir T%
AUPRC
HiE
One-hot LR
(0.00100118)
0.205 (0.041)
0.395 (0.016)
0.345 (0.056)
0.247 (0.019)
0.225 (0.104)
0.329 (0.026)
0.397 (0.036)
MarComp
0.219 (0.018)
0.191 (0.048)
0.395 (0.016)
0.356 (0.034)
0.240 (0.008)
0.225 (0.043)
0.342 (0.023)
0.345 (0.038)
scGREAT
0.224 (0.029)
0.222 (0.046)
0.416 (0.020)
0.433 (0.047)
0.245 (0.017)
0.183 (0.035)
0.393 (0.027)
0.390 (0.066)
GENELink
0.201 (0.020)
0.144 (0.050)
0.415 (0.016)
0.428 (0.060)
0.249 (0.013)
0.207 (0.054)
0.381 (0.030)
0.454 (0.094)
SCENIC:
0.210 (0.020)
0.200 (0.037)
0.465 (0.020)
0.568 (0.047)
0.227 (0.014)
0.219 (0.062)
0.346 (0.024)
0.393 (0.045)
GRNBoos12
0.218 (0.018)
0.162 (0.037)
0.440 (0.014)
0.553 (0.058)
0.226 (0.009)
0.167 (0.053)
0.356 (0.023)
0.348 (0.049)
DeepSEM
0.265 (0.032)
0.419 (0.059)
0,435 (0.019)
0.517 (0.043)
0.277 (0.014)
0.292 (0.095)
0.343 (0.024)
0.369 (0.048)
InloSEM-B (Ours)
0,331 (0.055)
0.547 (0.091)
0.498 (0.020)
0.533 (0.048)
0.298 (0.028)
0.472 (0.076)
0,388 (0.023)
0.522 (0.047)
InfoSEM-BC (Ours)
0.331 (0.056)
0.585 (0.094)
0.499 (0.020)
0,550 (0.053)
0.322 (0.032)
0.498 (0.069)
0.408 (0.020)
0.575 (0.045)
Random
0.222
0.215
0.390
0.397
0.232
0.231
0.349
0.351
Table The GRN inference performance, measured by AUPRC and Hit@ 1%, with corresponding standard error of the mean (in
parentheses) for each method evaluated on interactions between unseen genes with cell-type specific ChiP-seq targets. The top-2 best models are bold.
native loss (Eq.1).
AUPRC on hESC
Random
unseen genes
0.3
Run-time analysis: We include comparison of run times between InfoSEM and other baselines on different datasets with various numbers of cells and genes in Appendix B. Our results demonstrate that InfoSEM is faster than popular supervised learning baselines and only slightly slower than DeepSEM, while consistently outperforming them in terms of performance in unseen genes benchmark.
0.2
0.1
0.0
DeepSEM
InfoSEM-B (onehol)
InfoSEM-B
InfosEM-B (BioBERT)
InfoSEM-B (GO)
InfeSEM-B
(randam)
(ScBERT)
AUPRC on hHEP
Sensitivity analysis: Since InfoSEM-B uses embeddings from BioBERT, it is natural to ask: how useful are these embeddings in improving GRN inference? To explore this, we investigate the impact of alternative gene embeddings, including textual gene embeddings from another language model GenePT (Chen & Zou, 2024), gene embeddings de- rived from gene ontology (GO) (Ashburner et al., 2000), and gene embeddings from a single cell foundation model scBERT (Yang et al., 2022). Figure 4 shows that InfoSEM- B models replace BioBERT embeddings with other gene embeddings, such as InfoSEM-B (GenePT) and InfoSEM-B -B (GO), still resulting in improved GRN inference compared to De As expected, using non-informative embed- dings, such as one-hot or random embeddings, in InfoSEM- B does not enhance performance. Interestingly, we observe using embeddings from scBERT does not improve perfor- mance. We hypothesize that this may result from the binning of scRNA-seq data in scBERT, which could lead to loss of important information. Additionally, embeddings from ScBERT, derived from binned scRNA-seq data, are similar to those used in InfoSEM already and may not provide com- plementary insights for the adjacency matrix compared to embeddings like gene ontology or BioBERT, which are de- rived independent of scRNA-seq data. We believe, though, that exploring other scRNA-seq foundation model derived gene embeddings presents an interesting avenue for future exploration.
unseen genes
0.4
0.2
0.0
DeepSEM
InfoSEM.B
InfoSEM.B (random)
InfoSEM-8 (GenaPT)
InfoSEM.8 88
InfoSEM.B (SCBERT)
(onehot)
(GO)
Figure 4. Results of using different prior gene embeddings in InfoSEM-B with cell-type specifie ChIP-seq targets. InfoSEM-B with informative gene embedding, c.g., from BioBERT, GenePT, and gene ontology (GO), are better than models with noninforma- tive embeddings, e.g., one-hot and random. However, InfoSEM-B with gene embeddings from scBERT does not show significant improvement.
InfoSEM-BC which incorporates information from known interactions achieves the best performance on all datasets with respect to AUPRC and on out of datasets with re- spect to Hit@ 1 % for both cell-speeific and -ell--iiccci ChIP-seq target on unseen genes test set with average im- provements over DeepSEM by 31% and 66% (see Table 2 and Appendix Tables 6 and 7). By integrating prior knowl- edge from known interactions, InfoSEM-BC reconstructs scRNA-seg data effectively, enabling strong performance on unseen genes without the pitfalls of class imbalance exploited by supervised methods relying on their discrimi-
Additionally, we evaluate the sensitivity of GRN inference accuracy with respect to the number of cells the train-
8

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
ing data, detailed in Appendix F.3. We find that both InfoSEM-B and InfoSEM-BC, trained on just 20% of cells, achieve performance comparable to DeepSEM trained all cells for the hESC cell line.
precise diagnostics and targeted therapies. However, re- liance biases in existing datasets and benchmarks datasets may introduce artifacts that misrepresent true regulatory in- teractions, potentially leading to incorrect conclusions in downstream analyses. Traasparency in model evaluation and careful benchmarking, as emphasized in our work, are essential to mitigate such risks.
6. Conclusion and discussion
In this work, we study the problem of Gene Regulatory Network (GRN) inference using scRNA-seq data. Existing benchmarks focus interactions between seen genes, i.e., all genes appear in both training and test sets. While suitable for predicting novel interactions within well-characterized gene sets, they fail to address real-world applications such as biomarker expansion (Lotfi Shahreza et al., 2018), where models must generalize to interactions between unseen genes with no prior knowledge about their interactions. Ad- ditionally, we show that the high performance of supervised methods even on existing benchmarks may be influenced by dataset (gene-specific biases, such as class imbalance, rather than their ability to learn true biological mechanisms.
Additionally, as GRN models improve in predictive accu- racy, their impact on biomedical research and healthcare decisions will grow. It is crucial to ensure that these models are evaluated rigorously and deployed responsibly, avoid- ing iverpinterpretttion of predictions and ensuring their applicability across diverse biological contexts. Ethical con- siderations should guide their use, promoting transparency, reproducibility, and equitable advancements in genomies and medicine.
References
Ahmed, K. T., Park, S., Jiang, Q., Yeu, Y., Hwang, T., and Zhang, W. Network-based drug sensitivity prediction. BMC medical genomics, 13:1-10, 2020.
To address this gap, we propose a new, biologically mo- tivated benchmarking framework that evaluates a model's ability to infer interactions between unseen genes. We also introduce InfoSEM, unsupervised generative model that integrates biologically meaningful priors by leveraging tex- tual gene embeddings from BioBERT. InfoSEM achieves 38.5% on average improvement over existing state-of-the art models without informative priors for GRN inference. Furthermore, we show that InfoSEM can incorporate known interaction labels when available, further enhancing perfor- mance by 11.1% average across datasets while avoiding the pitfalls of training with class imbalance.
Anonymous. Deciphering cell lineage gene regulatory network via MTGRN. In Submitted to The Thirteenth International Conference on Learning Representations, 2024. URL https: ptttps://openrewee// /oppnrevieweeeeeeeeeeeeeeeeeeeee / forum? id-Ecb6HBoolz under review.
Ashburner, M., Ball, C. A., Blake, J. A., Botstein, D., Butler, H., Cherry, J. M., Davis, A. P., Dolinski, K., Dwight, S. S., Eppig. J. T., et al. Gene ontology: tool for the unification of biology. Nature Genetics, 25(1): 29, 2000.
We believe InfoSEM's s ability to generalize to unseen gene interactions, even with no or minimal labeled data, makes it a powerful tool for real-world applications, such as target discovery/proortiizaion. It can also serve key compo- nent in active learning frameworks, guiding the acquisition of new interactions from wet-lab experiments and expanding our understanding of GRNs in practical scenarios.
Atanackovic, L., Tong, A., Wang, B., Lee, L. J., Bengio, Y., and Hartford, S. Dyngfn: Towards bayesian inference of gene regulatory networks with gflownets. Advances in Neural Information Processing Systems, 36, 2024.
Basso, K., Margolin, A. A., Stolovitzky, G., Klein, U., Dalla- Favera, R., and Califano, A. Reverse engineering of regulatory networks in human b cells. Nature genetics, 37(4):382-390, 2005.
Impact Statement
Understanding gene regulatory networks (GRNs) is funda- mental deciphering cellular processes, with broad appli- cations in disease research, drug discovery, and biomarker identification. Our work contributes to this field by intro- ducing more rigorous evaluation framework for GRN inference, emphasizing generalization to previously unseen genes-a a critical challenge in real-world biological studies.
Camp, J. G., Sekine, K., Gerber, T., Loeffler- Wirth, H., Binder, H., Gac, M., Kanton, S., Kageyama, J., Damm, G., Seehofer, D.,, et al. Multilineage communication regulates human liver bud development from pluripotency. Nature, 546(7659):533 533-5388 2017.
From an ethical standpoint, advances in GRN inference earry both opportunities and risks. Improved inference meth- ods accelerate biomedical research, leading to more
Chan, T. E., Stumpf, M. P., and Babtie, A. C. Gene reg- ulatory network inference from single-cell data using multivariate information measures. Cell systems, 5(3): 251-267, 2017.
9

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
Chen, G. and Liu, Z.PP. Graph attention network for link prediction of gene regulations from single-cell ma- sequencing data. Bioinformatics, 38(19):452--4529 2022.
Kommu, S., Wang, Y., Wang, Y., and Wang. X. Gene regu- latory network inference from pre-trained single-cell tran- scriptomics transformer with joint graph learning. arXiv preprint arXiv:2407.18881 2024.
Chen, J., Cheong, C., Lan, L.., Zhou, X., Liu, J., Lyu, A., Cheung, W. K., and Zhang, L. Deepdrim: deep neural network to reconstruct cell-type-specific gene regulatory network using single-cell ma-seq data. Briefings in bioin- formatics, 22(6):bba 2021.
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., and Kang. J. Biobert: pre-trained biomedical language representation model for biomedical text mining. Bioin- formatics, 36(4): (234):1234-120,, 2020.
Li, J., Lai, Y., Zhang, C., and Zhang. Q. Tgena: temporal gene coexpression network analysis using a low-rank plus sparse framework. Journal of Applied Statistics, 47(6): 1064-1083, 2020.
Chen, Y. and Zou, J. Simple and effective embedding model for single-cell biology built from chatgpt. Nature Biomed- ical Engineering, 2024.
Lotfi Shahreza, M., Ghadiri, N., Mousavi, S. R., Varshosaz, J., and Green, J. R. A review of network-based ap- proaches to drug repositioning. Briefings in Bioinfor- matics, 19(5) 892, 2018.
Chevalley, M., Roohani, Y., Mehrjou, A., Leskovec, J., and Schwab, P. Causalbench: A large-scale benchmark for network inference from single-cell perturbation data. arXiv preprint 1rXXi:2210.172855 2022.
Mao, G., Pang, Z., Zuo, K., Wang, Q., Pei, X., Chen, X., and Liu, J. Predicting gene regulatory links from single-cell rma-seq data using graph neural networks. Briefings in Bioinformatics, 24(6):bbad414, 2023.
Dai, H., Ng, I., Luo, G., Spirtes, P., Stojanov, P., and Zhang, K. Gene regulatory network inference in the presence of dropouts: causal view. arXiv preprint arXiv:2403. 15500, 2024.
Margolin, A. A., Nemenman, I., Basso, K., Wiggins, C., Stolovitzky, G., Favera, R. D., and Califano, A. Aracne: an algorithm for the reconstruction gene regulatory networks a mammalian cellular context, In BMC Bioin- formatics, volume 7, pp. 1-15. Springer, 2006.
Dehmer, M., Mueller, L. A., and Emmert-Streib, F. Quan- titative network measures as biomarkers for classifying prostate cancer disease states: systems approach to diagnostic biomarkers. PloS one, 8(11):e77602, 2013.
Ghosh. S. and Basu, A. Network medicine in drug design: implications for neuroinflammation. Drug discovery to- day, 17111-1))600-6077 2012.
Moerman, T., Aibar Santos, S., Bravo González-Blan C., Simm, J., Moreau, Y., Aerts, J., and Aerts, Grnboost2 and arboreto: efficient and scalable inference of gene regulatory networks, Bioinformatics, 35(12):115-22611 2019.
Haury, A.-C., Mordelet, F,, Vera-Licona, P., and Vert, J.- P. Tigress: trustful inference of gene regulation using stability selection. BMC Systems Biology, 6:1-17, 2012.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cour- napeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-leam: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2836 2011.
Hayashi, T., Ozaki, H., Sasagawa, Y., Umeda, M., Danno, H., and Nikaido, I. Single-cell full- total ma quencing uncovers dynamics of recursive splicing and enhancer mas. Nature communications, 9(1):119, 2018.
Huynh-Thu, V. A., Irrthum, A., Wehenkel, L., and Geurts, P. Inferring regulatory networks from expression data using tree-based methods. PloS one, 5(9)6 2010.
Pratapa, A., Jalihal, A. P., Law, J. N., Bharadwaj. A., and Murali, T. Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nature Methods, 17(2):147-1555 2020.
Karlebach, G. and Shamir, R. Modelling and analysis of gene regulatory networks. Nature reviews Molecular cell biology, 7700)777-770, 2008. 2008.
Rubinsteyn, A. and Feldman, S. fancyimpute: An imputa- tion library for python. URL URLhttts::/giithbb com/ iskandr wwaanciipputte
Kim, S. Ppcor: an package for fast calculation to semi- partial correlation coefficients. Communications for sta- tistical applications and methods, 22(6):665, 2015.
Shalek, A. K., Satija, R., Shuga, J., Trombetta, J., Gennert, D., Lu, D., Chen, P., Gertner, R. S., Gaublomme, J. T. Yosef, N., et al. Single-cell ma-seq reveals dynamic paracrine control of cellular variation. Nature, 510(7505): 363-369, 2014.
Kingma, D. P. Auto-encoding variational bayes. arXiv
preprint arxiv: 131.. 14, 2013.
10

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
Shu, H., Zhou, J., Lian, Q., Li, H., Zhao, D., Zeng, J., and Ma, J. Modeling gene regulatory networks using neural network architectures. Nature Computational Science, (7):491-501, 2021.
Yuan, Y. and Bar-Joseph, Z. Deep learning for inferring gene relationships from single-cell expression data. Pro- ceedings of the National Academy of Sciences, 116(52);
27151 2019.
Skok Gibbs, C., Mahmood, O., Bonneau, R., and Cho, K. Pmff-gm: variational inference approach to single-cell gene regulatory network inference using probabilistic matrix factorization. Genome Biology, 25(1))88, 2024.
Zheng, X., Aragam, B., Ravikumar, P. K., and Xing, E. P.
Dags with tears: Continuous optimization for structure learning. Advances in neural information processing systems, 31, 2018.
Troyanskaya, O., Cantor, M., Sherlock, G., Brown, P., Hastie, T., Tibshirani, R., Botstein, D., and Altman, R. B. Missing value estimation methods for dna microarrays. Bioinformatics, 17(6):500-525, 2001.
Zhu, H. and Slonim, D. From noise knowledge: Diffu- sion probabilistic model-based neural inference of gene regulatory networks. Journal of Computational Biology. 31(11):1087-110 2024.
Wagner, A., Regev, A., and Yosef, N. Revealing the vectors of cellular identity with single-cell genomies. Nature biotechnology, 34(11):1145-116) 2016.
Wang, J., Chen, Y., and Zou, Q. Inferring gene regulatory network from single-cell transcriptomes with graph toencoder model. PLoS Genetics, 1919)xe000942, 2023.
Wang. Y., Chen, X., Zheng, 7.,, Huang. I.., Xie, W., Wang, F., Zhang, Z., and Wong, K.-C. scgreat: Transformer- based deep-language model for gene regulatory network inference from single-cell transcriptomics. Iscience, 27 (4), 2024.
Weighill, D., Guebila, M. B.,
C., Glass, K., Quackenbush, J., Platig, J., and Burkholz, Gene regu- latory network inference as relaxed graph matching. In Proceedings of the AAAI Conference Artificial Intelli- gence, volume 35, pp. 10263-10272, 2021.
Xu, J., Zhang, A.. Liu, F., and Zhang, X. Stgrs:: in- terpretable transformer- -based method for inferring gene regulatory networks from single-cell transcriptomic data. Bioinformatics, (9(4)bbtad16 2023.
Yang, F., Wang, W., Wang, F., Fang, Y., Tang, D., Huang. J., Lu. H., and Yao, J. scbert as a large-scale pretrained deep language model for cell type annotation of single-cell ma- seq data. Nature Machine Intelligence, 4(10):852 866, 2022.
Yevshin, I., Sharipov, R., Kolmykov, S., Kondrakhin, Y., and Kolpakov, F. Gtrd: database gene transcription regulation- 2009 update. Nucleic acids research, 47(D)): D100-D1O1 2019.
Yu, Y., Chen, J., Gao, T., and Yu, M. Dag-gnn: Dag struc- ture learning with graph neural networks. In Interna- tional Conference on Machine Learning, pp. 7154-7163. PMLR. 2019.
Yuan, Q. and Duren, Z. Inferring gene regulatory networks from single-cell multiome data using atlas-scale external data. Nature Biotechnology, pp. 1-11, 2024.

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
A. Derivation of ELBO for InfoSEM-B and InfoSEM-BC.
this section, we derive the evidence lower bound (ELBO) of InfoSEM with two proposed priors.
A.1. InfoSEM-B
p(XHH) HF)
= po(X| (1)
= =log] /
(X|2, A),(A(H, 20 (1,,,,))pmmm (w)))
qál (Z A)dZdA
(x AA)
(16)
polX|Z, A)p(AA (A|H, (w))
= log
4ol X AjdZ
20(2|X, A)
ZE (Z)X
[log Do
A)] log p( (||H, w) Pog (w0) KL (1)x(2 A))
=LIntosEM- (A, (A,0,
A.2. InfoSEM-BC
Y)
w) - =f -opo(X12) A @ (4
=log| /
pa(X| A ④ @
2 (0 1 oo
(x Oo(A')
=log po(X|2,A (.
20 ( Á ( 41))
(17)
4,(2 1 &
2B Ar |o o (o(A4
+ logpe w) + log +og + log (w)
-KL 1o (Z, A @ CoA (AAAA)))))
-IMGSSE Bc( (Ā A A 0 20 w)
B. Run-time analysis
(#cell, #genc)
scGREAT
GENELink
SCENIC
GRNBoost2
DeepSEM
InfoSEM-B
InfoSEM-BC
(454, 844)
236.30
177.25
80.81
9.27
91.88
118.00
125.91
(758, 844)
239.96
172.72
97.74
10.81
112.55
134.11
164.16
(758, 1291)
357,52
258.91
142.51
13.74
116.01
206.86
211.37
Table Run-time analysis
We include comparison of run times (in seconds) for different configurations of the datasets in terms of number of cells and genes, (ficell, #gene), for InfoSEM and other baselines in Table 3. Our results demonstrate that InfoSEM is faster than popular supervised learning baselines and only slightly slower than DeepSEM, while consistently outperforming them in terms of performance in unseen genes benchmark.
12

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
C. Details of each dataset
C.I. Cell-type specific datasets
hESC
hHEP
mDC
mESC
number of genes
844
908
1216
1353
number of cells
758
425
383
421
number of positive links
4404
9684
1129
40083
number of negative links
23448
17556
24407
78981
averaged node degree of TFs
133.5
322.8
53.8
455.5
C.2. Non-cell-type specific datasets
hESC
ННEP
mDC
mESC
number of genes
844
908
1216
1353
number of cells number positive links
758
425
383
421
3318
4033
3694
7705
number of negative links
229626
279263
300306
678266
averaged node degree of TFs
12.0
12.9
14.8
15.2
D. Reproducibility: details and hyperparameters of each model
In general, we cross-validate hyper-parameteen using the partially known GRN on the training set ensure fair comparison with existing methods that the same strategy. Code and datasets will be made available acceptance.
One-hot LR: A logistic regression implemented by seikit-learn (Pedregosa al., 2011) with the La regularization coefficient cross-validated on the training set.
MatComp:
A matrix completion algorithm implemented by fancyimpute (Rubinsteyn & Feldman) with rank 128.
scGREAT:
Use hyperparameters and implementation provided by Wang et al. (2024).
GENELink: Use hyperparameters and implementation provided by Chen & Liu (2022).
DeepSEM:
Use hyperparameters and implementation provided by Yuan & Bar-Joseph (2019).
InfoSEM-B and InfoSEM-BC: We set hyper-parameters. c.g., neural network architectures, learning rates schedule, prior scale of latent variable Z, to be the same as DeepSEM (Yuan & Bar-Joseph, 2019). For unique hyper-parameten, of InfoSEM-B and InfoSEM-Bee BC, we cross-validate them on the training set using known GRN (shown as below).
Ground-truth
Methods
hESC
НHEР
mDC
MESC
Tao (InfoSEM-B, InfoSEM-BC)
0.1
10
10
cell-type specific ChIP-seq
(InfoSEM-BC)
0.1
V0.1
v0.1
0.1
h (InfoSEM-BC)
128
128
128
128
Tw (InfoSEM-B, InfoSEM-BC)
100
100
100
100
non-cell-type specific ChIP-seq
(InfoSEM-BC)
VO.1
0.1
v
v0.1
h (InfoSEM-BC)
128
128
128
128
13

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
E. Performance of supervised methods with downsampling
AUPRC on NESC
AUFFIC on NHEP
AUBPC on mDC
AUPRC on mESC
0.6
0.0
0.4
5.10
0.4
Arcodaa
arooued
Deggou
Deacon
Balenoul Cp
Gittles
Sal
0.3
0.1
0.2
0.0
0.3
0.0
0.0
Figure 5. AUPRC with corresponding standard of the of GRN inference models with cell-type specific target interactions between seen genes test sets, Although we apply downsampling to remove the class imbalance level associated with each TF, supervised methods still show inflated accuracy on most of cell lines using the class imbalance level associated with cach TG.
If the class imbalance level associated with each gene causes the inflated accuracy of supervised method, one straightforward solution to remove the class imbalance with downsampling. Specifically, we randomly select the same number of negative edges as the number of positive edges for each TF when training supervised model, and the performance shown Fig 5.
We observe that supervised methods still have inflated accuracy by comparing their performance on unseen interactions between seen genes and interactions between unseen genes. One reason is that edges associated with TGs can still be imbalanced even the edges associated with TFs are balanced, and supervised learning methods can still make use of such information easily.
Moreover, removing the class imbalance of TFs does not improve the performance on interactions between unseen genes and unsupervised methods, especially DeepSEM, are still outperform supervised methods on unseen genes.
14

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
F. Additional experimental results
F.I. Performance of all methods on unseen interaction between seen genes test set.
bESC
hHEp
mDC
MESC
AUPRC
Hit@1
AUPRC
Hit@ 1%
AUPRC
Hir@ 1%
AUPRC
Hit@ 1%
One-hol LR
0.600 (0.028)
0.932 (0.039)
0.835 (0.005)
1,000 (0.003)
0.176 (0.006)
0,250 (0.024)
0.844 (0.009)
0.991 (0.004)
MarComp
0.638 (0.026)
0.913 (0.037)
0.840 (0.006)
1.000 (0.006)
0.351 (0.013)
0.419 (0.034)
0.860 (0.007)
0.974 (0.008)
scGREAT
0.642 (0.029)
0.966 (0.039)
0.847 (0.006)
1.000 (0.000)
0.249 (0.011)
0.325 (0.047)
0.858 (0.009)
1.000 (0.001)
GENELink
0.565 (0.029)
0.964 (0.043)
0.782 (0.007)
0.942 (0.015)
0.178 (0.006)
0.162 (0.026)
0.690 (0.016)
0.716 (0.101)
SCENIC
0.177 (0.008)
0.176 (0.031)
0.396 (0.010)
0.449 (0.031)
0.119 (0.008)
0.171 (0.044)
0.328 (0.012)
0,380 (0.026)
GRNBoost2
0.173 (0.006)
0.230 (0.017)
0.384 (0.007)
0,423 (0.015)
0.115 (0.007)
0.141 (0.036)
0.352 (0.011)
0.342 (0.024)
DeepSFM
0,216 (0.010)
0.318 (0.016)
0.424 (0.011)
0.509 (0.028)
0.118 (0.006)
065 (0.033)
0.340 (0.013)
0.431 (0.023)
InfoSEM-B
0.374 (0.027)
0.752 (0.067)
0.467 (0.01
0.538 (0.020)
0.130 (0.010)
0.174 (0.048)
0.401 (0.012)
0.562 (0.016)
InfoSEM-BC
0,553 (0.028)
0,913 (0.053)
0.702 (0.007)
0.865 (0.023)
0.285 (0.016)
0.555 (0.063)
0.672 (0.010)
0.918 (0.026)
Random
0.168
0.171
0.383
0.384
0.110
0.111
0.346
0.344
Table 4. The GRN inference performance for each method evaluated on unseen interactions between seen genes with cell-type specific ChIP-seq targets. The top-2 best models are bold. We observe that all supervised learning methods, including one-hot LR, achieve much better performance than unsupervised methods.
hESC
hHEP
mDC
MESC
AUPRC
Hit@1%
AUPRO
Hit@ 1%
AUPRC
Hit@ 1%
AUPRC
Hit@ 1%
One-hot LR
0.159 (0.001)
0.257 (0.013)
0.186 (0.004)
0,306 (0.008)
0.126 (0.004)
0.228 (0.007)
0.127 (0.004)
0.205 (0.006)
MatComp
0.207 (0.017)
0.334 (0.020)
0.247 (0.012)
0.375 (0.016)
0.396 (0.009)
0.563 (0.015)
0,260 (0.009)
0.375 (0.013)
scGREAT
0.173 (0.002)
0.288 (0.014)
0.244 (0.006)
0.366 (0.004)
0.183 (0.012)
0,314 (0.004)
0.139 (0.004)
0.235 (0.004)
GENELink
0.059 (0.011)
0.101 (0.022)
0.088 (0.011)
0.157 (0.022)
0.103 (0.007)
0,141 (0.019)
0.025 (0.005)
0.025 (0.013)
SCENIC
0.019 (0.001)
0.033 (0.004)
0.019 (0.001)
0.036 (0.003)
0.020 (0.001)
0.036 (0.004)
0.021 (0.001)
0.044 (0.003)
GRNBoost2
0.019 (0.001)
0.031 (0.004)
0.018 (0.001)
0.031 (0.003)
0.020 (0.001)
0.031 (0.005)
0.018 (0.001)
0.042 (0.003)
DeepSEM
0,023 (0.001)
0.034 (0.004)
0.021 (0.001)
0.035 (0.004)
0.021 (0.001)
0.033 (0.005)
0,021 (0.001)
0.044 (0.002)
InfoSEM-B
0.025 (0.001)
0.038 (0.005)
0.021 (0.001)
0.037 (0.002)
0.022 (0.001)
0,038 (0.003)
0.023 (0.000)
0.042 (0.002)
InfoSFM-BC
0.119 (0.008)
0.29 (0.018)
0.171 (0.008)
0.331 (0.011)
0.325 (0.010)
0.516 (0.016)
0.159 (0.006)
0.309 (0.007)
Random 0.019
0.017
0.018
0.017
0.018
0.018
0.014
0.014
Table 5. The GRN inference performance for each method evaluated on unxeen inteructions between seen genes with non-cell-type specific ChiP-seq targets. The top-2 best models are bold. We observe that InfoSEM-BC achieves top-2 performance three dataset.
We show the performance of all methods evaluated under current evaluation setup, i.e., unseen interactions between seen genes, in Table 4 and Table 5 for cell-type specific and non-cell-type specific GRNs. We observe that trivial baselines (one-hot LR and matrix completion) without using any gene expression data always achieve top-2 performance.
F.2. Performance of methods on interaction between unseen genes test set for non-cell-type specific ChIP-seq target.
We show the performance of all methods evaluated on interactions between unseen genes for the non-cell-type specific GRNs in Table 6, where the proposed InfoSEM is among top-2 best models in all cell lines. We observe that both AUPRC and Hit@ 1 % are very small when benchmarked against non-cell-type specific GRNs unlike cell specific GRNs in Table 2. One reason is that negative links in non-cell-type specific GRNs contain both unknown positive links and negative links due to the data collection process (Pratapa et al., 2020), therefore, negative links in non-cell-type specific GRNs much noisier than cell-type specific GRNs. However, positive links in non-cell-type specific GRNs do not contain such noise. We compute the recall to evaluate the performance of our methods on positive links only (Table 7) where we observe a much higher accuracy compared with Table 6 when both positive links and negative links are evaluated.
F.3. Sensitivity analysis w.r.f. the number of cells
In Figure 6, we illustrate how InfoSEM with different priors and DeepSEM perform on the unseen gene test sets when only a fraction of cells in the hESC dataset are used to train the model. We observe that InfoSEM-BC is only a slightly better than InfoSEM-B. B. Moreover, InfoSEM-B and InfoSEM-BC trained only on 20% cells can achieve a similar performance as DeepSEM trained on all cells.
15

---

InfoSEM: A Deep Generative Model with Informative Priors for Gene Regulatory Network Inference
bESC
ННEР
MDC
mESC
AUPRC
Hit@ 1%
AUPRC
Hit@ 1%
AUPRC
Hit@ 1%
AUPRC
Hit@ 1%
One-hot LR
0.025 (0.002)
0.029 (0.008)
0.022 (0.002)
0.006 (0.006)
0.022 (0.002)
0.012 (0.004)
0.014 (0.001)
0.008 (0.004)
MatComp
0.024 (0.001)
0.017 (0.004)
0.021 (0.001)
0.017 (0.003)
0.023 (0.001)
0.031 (0.005)
0.014 (0.000)
0.014 (0.003)
scGREAT
0.025 (0.003)
0.031 (0.011)
0.029 (0.003)
0.027 (0.010)
0.028 (0.002)
0.024 (0.006)
0.024 (0.001)
0.037 (0.006)
GENELink
0.023 (0.001)
0.006 (0.006)
0.020 (0.002)
0.014 (0.007)
0.026 (0.002)
0.028 (0.013)
0.015 (0.001)
0.012 (0.003)
SCENIC
0.020 (0.001)
0.030 (0.007)
0.025 (0.002)
0.044 (0.008)
0.021 (0.001)
0.016 (0.003)
0.022 (0.003)
0.042 (0.009)
GRNBoost2
0.022 (0.001)
0.018 (0.006)
0.022 (0.001)
0.027 (0.009)
0.024 (0.001)
0.020 (0.004)
0.022 (0.003)
0.041 (0.009)
DeepSEM
0.028 (0.002)
0.032 (0.005)
0.026 (0.003)
0.028 (0.010)
0.026 (0.001)
0.028 (0.004)
0.022 (0.002)
0.028 (0.007)
InfoSEM-B (Ours)
0.036 (0.004)
0.038 (0.007)
0.029 (0.003)
0.049 (0.009)
0,050 (0.004)
0.077 (0.014)
0.023 (0.003)
0.032 (0.008)
InfoSEM-HB BC (Ours)
0.038 (0.004)
0.042 (9.006)
0.030 (0.003)
0.054 (0.007)
0.051 (0.004)
0.082 (0.014)
0.023 (0.002)
0.045 (0.007)
Random
0.024
0.024
0.021
0.021
0.024
0.022
0.014
0.014
Table The GRN inference performance for each method evaluated interactions between genes with non-cell-type specific
ChiP-seq targets. The top-2 best models are bold. We observe that InfoSEM-BB BBC achieves top-2 performance on all dataset and InfoSEM-B achieves top-2 three datasets.
hESC
ННЕР
mDC
mESC
DeePSEM
0.10 (0.04)
0.11 (0.04)
0.60 (0.03)
0.03 (0.02)
InfoSEM-B (Ours)
0.19 (0.09)
0.34 (0.14)
0.64 (0.
0.19 (0.12)
InfoSEM-BC (Ours)
0.22 (0.09)
0.38 (0.14)
0.64 (0.10)
0.21 (0.11)
Table Averaged recall each method with non-cell-type specific target GRNs using threshold
Sensitivity analysis on hESC with specific ChiP-seq targets
DeepSEM
0.375
InfoSEM-B
InfoSEM-BC
0.350
0.325
AUPRC
0.300
0.275
0.250
0.225
0.200
0.2
0.4
0.6
0.8
1.0
Fraction of number of cells
Figure AUPRC models on unseen genes test set different numbers cells the training data.
16


Reviews:
[{"review_id": "k04GYdoNcw", "reviewer": "Reviewer_bvmZ", "summary": "The paper introduces InfoSEM, an unsupervised generative model for Gene Regulatory Network (GRN) inference that leverages textual gene embeddings as informative priors. The model can also integrate ground truth (GT) labels when available, avoiding biases and enhancing performance. The authors propose a new benchmarking framework that evaluates interactions between unseen genes, better aligning with real-world applications like biomarker discovery. The paper also highlights limitations in existing supervised learning benchmarks, showing that supervised models may exploit dataset biases rather than capturing true biological mechanisms.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s valuable feedback and acknowledgment of our work’s novelty and rigor. We are happy that our analysis of biases in supervised GRN benchmarks and InfoSEM’s performance on the unseen-gene benchmark were well received. Below, we address the reviewer’s questions and provide additional results, which we will include in the camera-ready version of the paper.\n\n**1.Evaluation in Traditional Seen-Gene Setting**: Thank you for the important point! We now show InfoSEM's performance in this setting below for all datasets (cell-type-specific ChIP-seq targets).\n\n| Method     | hESC AUPRC | hESC Hit@1% | hESC AUPRC | hESC Hit@1% | mDC AUPRC | mDC Hit@1% | mESC AUPRC | mESC Hit@1% |\n|------------|------------|-------------|------------|-------------|-----------|------------|------------|-------------|\n| One-hot LR | .600(.028) | .932(.039) | .835(.005) | 1.(.003) | .176(.006) | .250(.024) | .844(.009) | .991(.004)|\n| MatComp    | .638(.026) | .913(.037) | .840(.006) | 1.(.006) | .351(.013) | .419(.034) | .860(.007) | .974(.008) |\n| DeepSEM    | .216(.010) | .318(.016) | .424(.011) | .509(.028) | .118(.006) | .165(.033) | .340(.013) | .431(.023) |\n| InfoSEM-B  | .374(.027) | .752(.067) | .467(.011) | .538(.020) | .130(.010) | .174(.048) | .401(.012) | .562(.016) |\n| InfoSEM-BC | .553(.028) | .913(.053) | .702(.007) | .865(.023) | .285(.016) | .555(.063) | .672(.010) | .918(.026) |\n\nThe results show that for traditional seen-gene benchmarks, simple supervised models (One-hot LR, MatComp) without utilizing scRNA-seq perform best by leveraging gene-specific dataset biases, a fact which our work highlights for the first time and is a key contribution of our work. Even in this context, InfoSEM-BC maintains a competitive balance, avoiding overfitting to the gene-specific biases while performing better across multiple datasets compared to other unsupervised methods (GRNBoost2, DeepSEM). However, we again emphasize that the key takeaway is for seen genes benchmark, simple supervised models (One-hot LR, MatComp) that exploit gene-specific biases from known interactions are sufficient even without scRNA-seq. Hence, we propose to evaluate more advanced machine learning models on unseen-gene benchmarks where simple models entirely fail as shown in paper.\n\n**2.Gene embeddings from knowledge graphs**: Thank you for the suggestion! Our flexible framework can incorporate embeddings from other sources, such as knowledge graphs. To showcase this, we now did additional experiments using Gene Ontology (GO)  knowledge graph embeddings as priors (AUPRC shown below on unseen-genes benchmark), which improve our InfoSEM-B (BioBERT) results even further on human cell lines but lag behind our original InfoSEM-B (BioBERT) on mouse cell lines. This is because GO knowledge graph used is specific to human cells. Unlike BioBERT, which is more generic, knowledge graphs have to be chosen carefully depending on the context.\n\n| Method  | hESC | hHEP| mDC | mESC |\n|---------|------|------|------|------|\n| DeepSEM | .265(.032) | .435(.019)| .277(.014)| .343(.024)|\n| Our InfoSEM-B (one-hot)| .270(.038) | .447(.027)| .224(.012)| .338(.022)|\n| Our InfoSEM-B (BioBERT)| .332(.055) | .498(.020)| **.298(.028)**| **.388(.023)**|\n| Our InfoSEM-B (GO)| **.345(.057)** | **.524(.034)**| .275(.011)| .359(.024)|\n\n**3.Compute Efficiency**: We have now included comparison of run times (in seconds) for different configurations of (num_cell, num_gene) for InfoSEM and other baselines. Our results demonstrate that InfoSEM is faster than popular supervised learning baselines and only slightly slower than DeepSEM, while consistently outperforming them in terms of performance in unseen genes benchmark.\n\n| Model     |(454, 844) | (758, 844) | (758, 1291) |\n|-----------|-----------|------------|-------------|\n| scGREAT| 236.30 | 239.96| 357.52 |\n| GENELink| 177.25 | 172.72 | 258.91|\n| SCENIC|  80.81 | 97.74| 142.51|\n| GRNBoost2 |   9.27 | 10.81| 13.74|\n| DeepSEM   |  91.88 | 112.55| 116.01|\n| InfoSEM-B | 118.00 | 134.11| 206.86|\n| InfoSEM-BC| 125.91 | 164.16| 211.37|\n\n**4.Rationale for **Aᵉ** and **Aˡ** splitting of interaction matrix**:\nBy splitting the interaction matrix into two components—**Aᵉ** for the magnitude and **Aˡ** for the logit—we create a setup where both components can incorporate prior biological knowledge independently so that any possible misspecification of one prior will not affect the other. The prior on **Aᵉ**, informed by embeddings, acts as a starting point, but the model can adjust its MAP estimate based on data. Similarly, the prior on **Aˡ** allows for a probabilistic treatment of known interactions, but leaves room for learning from the data where these interactions might not be fully observed.\n\nWe hope these clarifications and new results address the reviewer’s concerns and enhance our paper's contributions. We kindly request consideration for a score increase based on this. We are happy to address any further suggestions!", "claims_and_evidence": "The authors claim that current GRN inference benchmarks fail to generalize well because they are trained and evaluated on the same set of genes, leading to an overestimation of model performance. This claim is supported by experiments where supervised models show significant performance drops when evaluated on unseen genes.\n\nHowever, a major limitation is that the proposed method is only evaluated on the new unseen-gene benchmark, and it does not compare performance in the traditional setting where models infer unseen regulatory interactions among known genes (seen-gene sets). Given that many real-world biological applications prioritize discovering novel interactions among well-studied genes, the broader impact of InfoSEM’s approach may be limited.", "methods_and_evaluation_criteria": "The evaluation methodology is well-structured, incorporating multiple datasets from the BEELINE suite and relevant baselines (e.g., DeepSEM).\n\nHowever, the paper only evaluates InfoSEM on the new unseen-gene benchmark and does not compare its performance in the traditional seen-gene setting. This omission raises concerns about whether InfoSEM is broadly applicable to all GRN inference tasks.\n\nThe rationale for selecting BioBERT embeddings as informative priors is reasonable, but it would be helpful to compare against other gene representation techniques (e.g., knowledge graph embeddings).", "theoretical_claims": "The variational Bayes framework for InfoSEM appears mathematically sound.\nThe decomposition of the adjacency matrix (Ae and Al) into effect size and interaction probability is novel, but the paper does not provide strong theoretical justification for why this decomposition improves GRN inference.", "experimental_designs_or_analyses": "The experiments convincingly demonstrate that supervised models overfit seen genes and fail to generalize to unseen genes.\n\nHowever, the paper does not compare InfoSEM’s performance for discovering unseen interactions among seen genes, which is a key limitation. In many real-world settings, researchers seek to uncover new regulatory interactions within well-characterized gene networks, rather than predicting interactions for entirely unseen genes.\n\nComputational efficiency is not addressed, which is critical for scaling deep generative models to larger genomic datasets.", "relation_to_broader_scientific_literature": "The paper is well-positioned in the GRN inference and single-cell genomics literature and aligns with recent efforts to develop interpretable AI models for biology.\nHowever, the focus on unseen-gene inference is not well contextualized within broader biological research, where many studies aim to infer novel regulatory interactions among well-characterized genes.", "essential_references_not_discussed": "The paper should discuss related work in knowledge graph-based GRN inference, as BioBERT embeddings are not the only way to encode biological prior knowledge.", "other_strengths_and_weaknesses": "Strengths:\nNovel use of BioBERT embeddings as informative priors for GRN inference.\nInsightful critique of dataset biases in supervised GRN benchmarks.\nDemonstrates strong performance on the new unseen-gene benchmark.\n\nWeakness:\nDoes not evaluate InfoSEM in the traditional seen-gene setting, limiting its applicability to broader GRN inference tasks.\nComputational efficiency is not addressed, which is important for scalability.\nLack of theoretical justification for the adjacency matrix decomposition (Ae and Al).", "other_comments_or_suggestions": "The authors should evaluate InfoSEM’s performance in the seen-gene setting to determine whether it is broadly applicable.\nA runtime comparison of InfoSEM vs. baselines would improve understanding of its computational feasibility.\nA more explicit discussion of the real-world implications of predicting interactions among unseen genes vs. seen genes would strengthen the paper.", "questions_for_authors": "1.\tWhy is InfoSEM not evaluated in the traditional seen-gene setting? This is a crucial benchmark for real-world GRN inference.\n2.\tHow does InfoSEM compare computationally to other deep generative models for GRN inference?\n3.\tWould using alternative gene embeddings (e.g., knowledge graph embeddings) affect performance?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ul53aFUXgA", "reviewer": "Reviewer_TUBJ", "summary": "Summary\n\nThis study proposes infoSEM, a deep generative model with informative priors for gene regulatory network (GRN) inference. By integrating text-based gene embeddings as biological priors, it addresses the critical challenge of GRN reconstruction without ground-truth interaction labels. The authors also introduce a novel benchmarking framework to evaluate predictions for unseen gene relationships.\n\nMain findings:\n\n1. Leveraging textual gene embeddings as informative priors enhances GRN inference when interaction labels are unavailable.\n\n2. Further integrate available interaction labels as additional priors to reduce bias and improve performance.\n\nMain Results:\n\n1. Evaluated the performance of existing supervised and unsupervised methods under established benchmarks, analyzing why simple methods like one-hot logistic regression (LR) perform remarkably well.\n\n2. Supervised models (e.g., scGREAT and GENELink) showed significant performance degradation in predicting interactions between unseen genes, while unsupervised methods (especially DeepSEM) outperformed supervised methods across three datasets.\n\n3. The InfoSEM-B model improves GRN inference by 25% and 52% on average over DeepSEM when gene interactions are unknown. The InfoSEM-BC variant, which incorporates known interaction information, achieves the best AUPRC performance across all datasets.\n\n4. The study also demonstrates that BioBERT embeddings enhance GRN inference.\n\n\nMain algorithmic ideas:\n\nBuilt upon the DeepSEM framework, integrating BioBERT embeddings and gene interaction embeddings as prior knowledge.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback and acknowledgment of our real-world benchmark as a key advancement in GRN inference, along with their praise for our theoretical clarity. Below, we address their questions and provide additional results, which we will include in the camera-ready version.\n\n**1. Ablation studies (BioBERT embeddings+LR, AE loss)**:\nThank you for the suggestion! We have now tested BioBERT embeddings in LR on unseen genes benchmark. Results below show that BioBERT+LR performs similarly or slightly better than the 1-hot LR baseline in paper. This is expected, as in the unseen-gene setting, both LR baselines (with 1-hot or BioBERT) can't leverage gene-specific biases for GRN inference. The slight improvement with BioBERT embeddings highlights the informativeness of the embeddings over 1-hot encoding. However, both these baselines perform considerably worse compared to our InfoSEM-B/InfoSEM-BC models, underscoring the benefits of our proposed framework as a whole beyond simply incorporating BioBERT as prior.\n\nWe explored the AE loss function (DeepSEM-AE) suggested by the reviewer, but it performed significantly worse than DeepSEM and our InfoSEM-B and InfoSEM-BC models due to DeepSEM's assumption of a Gaussian distribution for **Z** [1, 2], which requires regularization in the variational Bayes loss. Our InfoSEM-B and InfoSEM-BC naturally serve as ablation studies on the baseline DeepSEM. InfoSEM-B serves as an ablation study by replacing DeepSEM's Laplace prior over A with our informative prior, while InfoSEM-BC adds **Aˡ** and its informative prior to InfoSEM-B. We are open to any further ablation studies if the reviewer suggests.\n\n|Method|hESC AUPRC|hESC Hit@1%|hHEP AUPRC|hHEP Hit@1%|mDC AUPRC|mDC Hit@1%|mESC AUPRC|mESC Hit@1%|\n|------------|------------|-------------|------------|-------------|-----------|------------|------------|-------------|\n| 1-hot LR |.210(.018)| .205(.041)| .395(.016)| .345(.056)| .247(.019)| .225(.004)| .329(.026)| .397(.036)|\n| BioBERT+LR|.212(.019)| .227(.050)| .423(.023)| .450(.076)| .230(.020)| .151(.050)| .347(.021)| .397(.088)|\n| DeepSEM-AE|.203(.016)| .183(.033)| .400(.018)| .483(.054)| .211(.009)| .143(.022)| .321(.021)| .311(.031)|\n| DeepSEM| .265(.032)| .419(.059)| .435(.019)| .517(.043)| .277(.014)| .292(.095)| .343(.024)| .369(.048)|\n| InfoSEM-B (Ours)|**.331(.055)**| .547(.091)| **.498(.020)**| .533(.048)| .298(.028)| .472(.076)| .388(.023)| .522(.047)|\n| InfoSEM-BC (Ours)|**.331(.056)**| **.585(.094)**| **.499(.020)**| **.550(.053)**| **.322(.032)**| **.498(.069)**| **.408(.020)**| **.575(.045)**|\n\n**2. Evaluation on Original Benchmarks**:\nThank you for your comment! We have now included InfoSEM on the original seen-gene benchmarks in our response to reviewer bvmZ, with results presented in the accompanying table. Our results show that for traditional seen-gene benchmarks, simple supervised models (One-hot LR & MatComp) perform best by leveraging gene-specific biases without using scRNA-seq data, a fact which our work highlights for the first time and is a key contribution of our work. Even in this context, InfoSEM-BC maintains a competitive balance, avoiding overfitting gene-specific biases while performing better than unsupervised baselines. However, the key takeaway is that for seen-gene benchmarks, simpler models that exploit gene-specific biases are sufficient, and hence, we propose to evaluate advanced machine learning models on unseen-gene benchmarks.\n\n**3. Limited  architectural innovation over DeepSEM**:\nWe emphasize that our contribution is not on architectural innovation over DeepSEM, but in extending it within a principled variational Bayes framework to leverage informative priors (e.g., gene embeddings from language models) or available ground truth information without overfitting to gene-specific biases, an aspect not explored so far. Additionally, we show for the first time that existing supervised learning models tend to leverage gene-specific biases in the dataset, and hence, we introduce a new unseen genes benchmark to provide a reliable testbed for evaluating generalizability of GRN inference methods. As the reviewer themselves noted, our benchmark for evaluating generalizability to unseen gene relationships is a crucial advancement for GRN inference, and we are happy that it has been well received.\n\n**4. Including citations to proposed references**: We thank the reviewer for additional references. We have already cited the first and third references in the paper and will include the second one in the camera-ready version too. These citations will enhance the discussion and provide further context to our work.\n\nWe hope that our clarifications and additional results address your concerns! We kindly request the reviewer to consider raising their score, taking these into account. \n\n#### References\n[1] DAG Structure Learning with Graph Neural Networks, 2019 \\\n[2] Modeling gene regulatory networks using neural network architectures, 2021", "claims_and_evidence": "1. Under existing benchmarks, the performance of current supervised and unsupervised methods is significantly inferior to the one-hot logistic regression (LR) approach. The minimal differences in AUPRC also were observed on the new benchmark dataset. A question arises: Would replacing one-hot encoding with BioBERT embeddings in LR yield improved performance on the new benchmark?\n\n\n2. InfoSEM-B and InfoSEM-BC were only evaluated on the new benchmark, with no assessment conducted on original benchmarks.", "methods_and_evaluation_criteria": "1. The proposed method demonstrates limited technical improvements compared to DeepSEM, primarily consisting of additional embedding incorporation.\n\n2. The newly proposed benchmark represents a significant advancement for GRN inference research, particularly in addressing the critical challenge of generalizability to unseen gene relationships.", "theoretical_claims": "1. The work provides detailed theoretical derivations and mathematical formulations, enabling readers to easily comprehend the mathematical principles and logical foundations underlying the model.\n\n2. Existing methodologies are thoroughly described and systematically categorized.", "experimental_designs_or_analyses": "The experimental design (unseen gene) is reasonable.\n\n\n\nExperimental analysis:\n\n1. InfoSEM-B and InfoSEM-BC only evaluate the results on the new benchmark, not the results on the original benchmark.\n\n2. Ablation studies should be supplemented, such as BioBert's dembedding + LR, ...", "relation_to_broader_scientific_literature": "This work built upon the DeepSEM framework, integrating BioBERT embeddings and gene interaction embeddings as prior knowledge.", "essential_references_not_discussed": "The paper adequately cites and discusses foundational works relevant to GRN inference, including:\n\n1. Supervised approaches(scGREAT, GENELink)\n2. Unsupervised variational approaches (DeepSEM, GRNBoost2)\n\nWhile the core references are sufficient, expanding the discussion to include:\n\n1. Gene Regulatory Network Inference from Pre-trained Single-Cell Transcriptomics Transformer with Joint Graph Learning. ICML 2024 AI for Science workshop\n\n2. Boosting single-cell gene regulatory network reconstruction via bulk-cell transcriptomic data. Briefings in Bioinformatics, 23(5):bbac389, 2022.\n\n3. STGRNS: an interpretable transformer-based method for inferring gene regulatory networks from single-cell transcriptomic data. Bioinformatics 2023", "other_strengths_and_weaknesses": "Strengths:\n\n1. Conducts systematic evaluation of supervised vs unsupervised methods across legacy and novel benchmarks\n\n2. Empirically demonstrates BioBERT's effectiveness as biological priors for GRN inference.\n\n\nWeaknesses:\n\n1. The proposed method shows limited architectural innovation over DeepSEM (primarily embedding additions)\n\n2. InfoSEM variants lack ablation studies", "other_comments_or_suggestions": "None", "questions_for_authors": "1. The proposed method demonstrates limited technical improvements compared to DeepSEM, primarily consisting of additional embedding incorporation.\n\n\n2. Under existing benchmarks, the performance of current supervised and unsupervised methods is significantly inferior to the one-hot logistic regression (LR) approach. The minimal differences in AUPRC also were observed on the new benchmark dataset. A question arises: Would replacing one-hot encoding with BioBERT embeddings in LR yield improved performance on the new benchmark?\n\n3. InfoSEM-B and InfoSEM-BC were only evaluated on the new benchmark, with no assessment conducted on original benchmarks.\n\n4. Ablation studies should be supplemented, such as different loss (VAE -> AE), and more other variants.\n\n5. While the core references are sufficient, expanding the discussion to include:\n[1] Gene Regulatory Network Inference from Pre-trained Single-Cell Transcriptomics Transformer with Joint Graph Learning. ICML 2024 AI for Science workshop\n[2] Boosting single-cell gene regulatory network reconstruction via bulk-cell transcriptomic data. Briefings in Bioinformatics, 23(5):bbac389, 2022.\n[3] STGRNS: an interpretable transformer-based method for inferring gene regulatory networks from single-cell transcriptomic data. Bioinformatics 2023", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fpbDxJBSzU", "reviewer": "Reviewer_rrUh", "summary": "This paper introduces a method that incorporates gene embeddings from pretrained language models or known gene-gene relationships into the existing DeepSEM framework, resulting in two models: InfoSEM-B and InfoSEM-BC. The authors provide a detailed discussion on how to integrate the interaction matrix $Y$ and adjacency matrix $A$ when introducing gene relationships. Additionally, they propose a new benchmark approach for evaluating gene regulatory network model performance, where not only are interactions absent from the training set, but transcription factors are also excluded from the training set.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful and constructive comments. We appreciate the recognition of our new benchmark and the value of integrating external priors for GRN inference. We address the reviewer’s questions below and will include them in the camera-ready version.\n\n**1. External Priors in Section 2.2**: Yes, the external priors refer to gene embeddings and gene relationships. These priors can be derived from various sources, including textual embeddings (e.g., BioBERT), knowledge graphs (e.g., Gene Ontology), or known gene interactions, all of which original DeepSEM cannot use directly.\n\n**2. Comparison with SCENIC**: We have now included a comparison with SCENIC, which demonstrates similar performance as GRNBoost2. Our InfoSEM-B and InfoSEM-BC outperform SCENIC across all cell lines for AUPRC and across all but hHEP for Hit@1% metric for the cell-type specific datasets.\n\n| Method| hESC AUPRC | hESC Hit@1% | hHEP AUPRC | hHEP Hit@1% | mDC AUPRC | mDC Hit@1% | mESC AUPRC | mESC Hit@1% |\n|-------|----------- |-------------|------------|-------------|-----------|------------|------------|-------------|\n| SCENIC| .210(.020)| .200(.037)| .465(.020)| **.568(.047)**| .227(.014)| .219(.062)| .346(.024)| .393(.045)|\n| InfoSEM-B (Ours)| **.331(.055)**| .547(.091)| **.498(.020)**| .533(.048)| .298(.028)| .472(.076)| .388(.023)| .522(.047)|\n| InfoSEM-BC (Ours)| **.331(.056)**| **.585(.094)**| **.499(.020)**|.550(.053)| **.322(.032)**| **.498(.069)**| **.408(.020)**| **.575(.045)**|\n\n**3. Reason for InfoSEM generalization to D_{unseen}**:  InfoSEM's main training objective is the gene expression reconstruction rather than the interaction prediction used in the supervised learning framework. This encourages the model to learn the gene-gene relationships from the gene expression data, instead of utilizing the gene-specific bias from the labels, hence contributing to its ability to generalize to the unseen gene set, D_{unseen}. Additionally, using gene embeddings as priors aids our model's generalization to the unseen gene set by leveraging those priors to learn gene relationships.\n\n**4. Use of Gene Embeddings from Single-cell Foundation Models**: We initially focused on gene embeddings from pretrained language models like BioBERT due to their effectiveness in providing semantic context for various models and tasks (see [1,2]). Furthermore, BioBERT is easier to use; it requires only the gene names already available, unlike single-cell models that necessitate specific preprocessing of scRNA-Seq data.\n\nFollowing the reviewer's suggestion, we experimented with embeddings from scBERT [3], a foundation model designed for single-cell RNA data, but did not observe significant improvements in performance over InfoSEM-B working with a one-hot baseline instead. We hypothesize that this may result from scBERT's binning of scRNA-Seq data, which could lead to a loss of important information. Additionally, scBERT's embeddings, derived from binned scRNA-Seq data, are similar to those used in InfoSEM already and may not provide complementary insights for the adjacency matrix compared to embeddings like Gene Ontology or BioBERT, which are derived independent of scRNA-Seq data. We believe, though, that exploring other scRNA-Seq foundation model derived gene embeddings presents an interesting avenue for future exploration.\n\nIn response to a similar query from reviewer bvmZ regarding the use of external priors from knowledge graphs, we conducted additional experiments using Gene Ontology (GO) embeddings as priors. The AUPRC results, shown below on the unseen-genes benchmark, demonstrate that this further enhances our InfoSEM results on human cell lines, reinforcing the importance of incorporating external priors from independent sources.\n\n| Method  | hESC AUPRC | hHEP AUPRC|\n|---------|------|------|\n| DeepSEM| .265(.032)| .435(.019)|\n| Our InfoSEM-B (onehot) | .270(.038)| .447(.027)|\n| Our InfoSEM-B (BioBERT) | .332(.055)| .498(.020)|\n| Our InfoSEM-B (GO) | **.345(.057)**|**.524(.034)**|\n| Our InfoSEM-B (scBERT) | .272(.029)| .442(.027)|\n\n**5. Data Leakage and Validation Measures**: Thank you for the important remark! Unlike seen-gene benchmarks, we ensured there was no data leakage between training and test sets in our proposed unseen benchmarking setup, as the genes between training and testing are different.\n\nWe hope our clarifications and additional experiments answer all your valuable comments. In light of the additional experiments and results presented, we kindly ask if the reviewer would consider raising their score. Thank you!\n\n#### References\n[1] scGREAT: Transformer-based deep-language model for gene regulatory network inference from single-cell transcriptomics, 2024 \\\n[2] GenePT: A Simple But Effective Foundation Model for Genes and Cells Built From ChatGPT, 2024 \\\n[3] scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data, 2022", "claims_and_evidence": "Yes, the authors support their claims through comparisons with baselines and through precise ablation experiments and relevant analyses.", "methods_and_evaluation_criteria": "Yes, AUPRC and Hit@1% appear to be reasonable metrics for evaluating the accuracy of gene regulatory network predictions.", "theoretical_claims": "I have examined the author's explanations regarding the variational inference proofs in the main text and did not find any issues.", "experimental_designs_or_analyses": "I have reviewed the performance comparison tables with baselines and the ablation experiment tables. The analysis results demonstrate the effectiveness of the author's proposed method and the effectiveness of using gene embeddings.", "relation_to_broader_scientific_literature": "The authors' proposed method is a direct extension of the DeepSEM[1] method published in 2021.\n\n[1] Modeling gene regulatory networks using neural network architectures", "essential_references_not_discussed": "See the Familiarity With Literature and Questions For Authors sections.", "other_strengths_and_weaknesses": "This paper appears to be an incremental improvement on DeepSEM. Nonetheless, I personally believe that the work of integrating external gene knowledge and gene relationships is quite important. The authors' experimental conclusions demonstrate the effectiveness of these biological priors in improving gene regulatory network prediction performance, which I think may inspire more work in this direction.", "other_comments_or_suggestions": "See Questions For Authors.", "questions_for_authors": "1. In Section 2.2, do the mentioned external priors specifically refer to the gene embeddings and gene relationships mentioned later? Could you provide a brief explanation here?\n2. Would the authors have time to add a comparison with the SCENIC method?\n3. How does the authors' proposed model generalize to the $D_\\text{unseen}$ set? Is it through gene embeddings and gene interactions?\n4. Why did you prioritize trying gene embeddings from pretrained language models? How effective would gene embeddings from RNA foundation models (such as scGPT, scFoundation, etc.) be?\n5. Did the authors consider the impact of data leakage when integrating external priors? Were any validation measures taken?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "td3jMpXxnZ", "reviewer": "Reviewer_TKSk", "summary": "This paper emphasizes the reason why unsupervised gene regulatory network inference lags in supervised ones is that it did not well utilize the prior knowledge. They followed the framework proposed by DeepSEM and proposed infoSEM to involve prior knowledge provided by BioBERT and logit probability. In addition, the paper argued that the previous benchmarks suffer from robustness and imbalanced classes. They improved it by making sure that genes in test sets are not in training sets. The effectiveness of InfoSEM is verified on real-world datasets with Ground-Truth provided by ENCODE and  ChIP-Atlas.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s insightful comments. Below, we address their questions.  \n\n**1. Do Functionally Similar Genes Share Causal Relationships?**: The assumption that functionally similar genes share causal relationships is well supported by biological literature. Evolutionary principles suggest that genes used for related functions often maintain similar regulatory architectures [1,2]. Mendelian randomization studies show that genes affecting related traits frequently share causal variants in lipid metabolism and coronary heart disease [3,4]. Pleiotropy analyses further demonstrate that cell-type-specific genes influence multiple neuropsychiatric traits via overlapping pathways, reinforcing the notion of shared causal roles among functionally related genes [5]. We will include this discussion and relevant citations in the camera-ready version.\n\n**2. Evaluation of the Output GRN**: We use the established BEELINE GRN evaluation dataset [7], which only evaluates interactions with recorded ground-truth labels, consistent with previous works [8,9].\n\n**3. Reliability of BioBERT-Derived Prior**: BioBERT has been shown to enhance scRNA-seq information for GRN inference in prior work [10]. In our InfoSEM framework, we treat BioBERT embeddings as an additional prior, with their influence adjustable through a cross-validated hyperparameter that allows the model to reduce their importance if deemed unreliable.  Still, to further validate the reliability of BioBERT embeddings in response to the reviewer's question, we performed Gene Ontology enrichment analysis on gene clusters derived from hierarchical clustering of the similarity matrix of BioBERT embeddings, controlling for false discovery rate (FDR). Analysis of 908 genes revealed several significant functional enrichments post-FDR correction (FDR < 0.05), notably in lipid transport (FDR=6.80e-12), DNA replication initiation (FDR=6.95e-07), and RNA polymerase II transcription (FDR=3.49e-07). The enrichment patterns displayed clear biological specificity; for instance, one cluster included APOA1, APOA2, APOA4, and APOB, enriched for cholesterol efflux (FDR=4.12e-14), while another cluster encompassed multiple MED genes related to transcription regulation. These robust enrichments strongly suggest that BioBERT embeddings capture meaningful biological relationships between genes.\n\n**4. Training Single Model for Different Cell Types**:  This appears to be a misunderstanding. We do not train a single model for multiple cell types. Like DeepSEM [8] and other baselines, we train separate models for each cell type to capture cell-type-specific regulatory interactions instead of assuming a universal gene regulatory network. \n\n**5. Handling Latent Variables & Network Asymmetry**: The impact of latent variables on inferred direction of GRNs is an important research area, as shown in studies like [6]. We will include a discussion of this in our paper and cite [6]/other relevant works. However, our study relies on established datasets with experimentally validated interactions from prior works [7], which we believe minimizes the impact of latent confounders. Addressing confounders comprehensively will be a non-trivial task that warrants a separate paper, beyond the scope of our work.\n\n**6. Alignment of Test Set Split and ChIP-seq Ground Truth**: Like question 2's response, we adhere to established literature by considering only known interactions from ChIP-seq databases [7,8,9] and splitting them into training and test sets.\n\n**7. Choice of 1000 Genes Instead of 5000**: We follow BEELINE's established preprocessing method [7], selecting the top 1000 highly expressed genes to enhance the quality of inferred networks and ensure comparability with prior methods [8,9].\n\nIn light of additional experiments & clarifications provided in response to reviewer's comments, we kindly request consideration on raising the score. We believe these clarifications address all the comments!\n\n#### Reference\n[1] Evaluating the potential role of pleiotropy in Mendelian randomization studies, 2018  \n[2] Inferring causality and functional significance of human coding DNA variants, 2012  \n[3] Diagnostics for Pleiotropy in Mendelian Randomization Studies: Global and Individual Tests for Direct Effects, 2018  \n[4] Pleiotropy-robust Mendelian randomization, 2018  \n[5] Selecting causal genes from genome-wide association studies via functionally-coherent subnetworks, 2016  \n[6] Gene Regulatory Network Inference in the Presence of Selection Bias and Latent Confounders, 2025  \n[7] Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data, 2020 \\\n[8] Modeling gene regulatory networks using neural network architectures, 2021 \\\n[9] Graph attention network for link prediction of gene regulations from single-cell RNA-sequencing data, 2022 \\\n[10] scGREAT: Transformer-based deep-language model for gene regulatory network inference from single-cell transcriptomics, 2024", "claims_and_evidence": "This paper relies on an important assumption that genes, which are similar in function, prefer to have similar causal relationships. I think this claim needs evidence to support it. I do not know if it is reasonable in biology.", "methods_and_evaluation_criteria": "I agree with that Chip-seq is used as ground truth to evaluate the performance. But ENCODE and ChIP-Atlas usually have limited records, how do you evaluate the output gene regulatory network?\n\n1. How can we guarantee that the prior knowledge from BioBERT is reliable? \n2. For gene regulatory network inference, if you train a model for different cells, how do you guarantee that the output network is suitable for all cells, as the expressed genes are different among different cells? \n3. If you consider the data generation process following e.q. (3), how to identify the direction of genes if there is a latent variable affecting them?", "theoretical_claims": "There are no theoretical claims", "experimental_designs_or_analyses": "Yes. I am curious about how to align the splitting of test sets and the ground truth in evaluation, as if you randomly split the training and test sets, some genes may not be reported by Chip-seq.\n\nUsually the sc-RNA seq datasets have around 5000 genes, why the dataset you choose all is around 1000 genes.", "relation_to_broader_scientific_literature": "Providing a way to involve prior knowledge for unsupervised learning.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "How to guarantee the asymmetry of gene regulatory network, especially, considering the confounding effect provided by latent variables?", "other_comments_or_suggestions": "No", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes an unsupervised generative model for gene regulatory network (GRN) inference incorporating textual gene embeddings from language models as informative priors, improving performance by integrating known gene interactions. A new benchmarking strategy for testing models on unseen genes is introduced, demonstrating that the method outperforms existing supervised and unsupervised methods on multiple real-world datasets.

All reviewers were recommending acceptance of the paper.
Decision: Accept (poster)