KINDLE: Knowledge-Guided Distillation for Prior-Free Gene Regulatory Network Inference
Authors: Rui Peng
Yuchen Lu
Qichen Sun
Yuxing Lu
Chi Zhang
Ziru Liu
Jinzhuo Wang
Venue: NeurIPS 2025 poster ()
Keywords: gene regulatory network
Primary area: machine_learning_for_sciences

Abstract: Gene regulatory network (GRN) inference serves as a cornerstone for deciphering cellular decision-making processes. Early approaches rely exclusively on gene expression data, thus their predictive power remain fundamentally constrained by the vast combinatorial space of potential gene-gene interactions. Subsequent methods integrate prior knowledge to mitigate this challenge by restricting the solution space to biologically plausible interactions. However, we argue that the effectiveness of these approaches is contingent upon the precision of prior information and the reduction in the search space will circumscribe the models' potential for novel biological discoveries. To address these limitations, we introduce KINDLE, a three-stage framework that decouples GRN inference from prior knowledge dependencies. KINDLE trains a teacher model that integrates prior knowledge with temporal gene expression dynamics and subsequently distills this encoded knowledge to a student model, enabling accurate GRN inference solely from expression data without access to any prior. KINDLE achieves state-of-the-art performance across four benchmark datasets. Notably, it successfully identifies key transcription factors governing mouse embryonic development and precisely characterizes their functional roles. In mouse hematopoietic stem cell data, KINDLE accurately predicts fate transition outcomes following knockout of two critical regulators (Gata1 and Spi1). These biological validations demonstrate our framework's dual capability in maintaining topological inference precision while preserving discovery potential for novel biological mechanisms.

KINDLE: Knowledge-Guided Distillation for Prior-Free Gene Regulatory Network Inference
Rui Pengl.? Yuchen Lu", Qichen Sun', Yuxing Lu', Chi Zhang', Ziru Liu', Jinzhuo Wang' 'Department of Big Data and Biomedical AI, College of Future Technology, Peking University CCenter for BioMed-X Research, Academy for Advanced Interdisciplinaryy Studies, Peking University
School of Physics, Peking University
+ *Yuanpei College, Peking University
{pengrui, luyuchen2002 200000000880 18200, luyx, cszc21, (ezrc2,,2rr)a (alz)@sstu pku. edu. wangj inzhuo@pku. hemppuu.ddueee
Abstract
Gene regulatory network (GRN) inference serves as a cornerstone for deciphering cellular decision-making processes. Early approaches rely exclusively on gene expression data, thus their predictive power remain fundamentally constrained by the vast combinatorial space of potential gene-gene interactions. Subsequent methods integrate prior knowledge to mitigate this challenge by restricting the solution space to biologically plausible interactions. However, argue that the effectiveness these approaches is contingent upon the precision of prior in- formation and the reduction in the search space will circumscrihe the models' potential for novel biological discoveries. To address these limitations, we in- troduce KINDLE, three-stage framework that decouples GRN inference from prior knowledge dependencies. KINDLE trains a teacher model that integrates prior knowledge with temporal gene expression dynamics and subsequently distills this encoded knowledge to student model, enabling accurate GRN inference solely from expression data without access to any prior. KINDLE achieves state- of-the-art performance across four benchmark datasets. Notably, it successfully identifies key transcription factors governing mouse embryonic development and precisely characterizes their functional roles. In mouse hematopoietic stem cell data, KINDLE accurately predicts fate transition outcomes following knockout two critical regulators (Gatal and Spi1). These biological validations demonstrate our framework's 's dual capability in maintaining topological inference precision while preserving discovery potential for novel biological mechanisms.
Introduction
Gene regulatory network (GRN) represents a directed graph that depicts the regulatory interactions between genes, where nodes consist of transcription factors (TFs) and target genes (TGs). A directed edge between a TF and TG signifies the TF's capacity to bind the cis-regulatory elements of the TG and subsequently modulates its transcriptional activity (I]. GRN provides mechanistic blueprints for understanding regulatory logic underlying lineage commitment, maintenance, and reprogramming Precisely resolved GRN enables mechanistic interpretations lineage bifurcation, aging process, and tumor-related dysregulation 3
Despite their biological significance, GRN inference remains technically challenging. Early inference methods that rely solely on gene expression data face inherent limitations: The explorable TF-TG
"Corresponiing Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

SC
validated M x (M
1)
M
M
+
K
Single-cell Expression Matrix (M genes)
Prior network (K edges)
Search Space
Figure I: For dataset with . genes, the search space for gene pairs spans M (M 1) possible interactions, as self-loop edges are not considered. Prior-based methods drastically narrow the exploration to the K edges supported by prior knowledge (K < M).
interaction space scales quadratically with the number of genes, resulting in approximately billion potential regulatory interactions within the whole genome (comprising approximate 30,000 genes). This vast search space fundamentally constrains the performance of expression- based approaches. Contemporary methods address this by incorporating prior knowledge from complementary data (e.g.. ScATAC-sea seq or Hi-C [5]) constrain the search space to pre-defined TF-TG pairs, as illustraed in Figure I Although prior-based approaches enhance performance by narrowing the search space, they impose two major limitations. Firstly, with a fixed prior network, an algorithm is confined to searching among its existing edges, and its performance depends on the overlap between the prior and the ground truth network. A perfect match allows for 100% accuracy, while minimal or no overlap leads to zero accuracy for all algorithms. Secondly, limiting candidate edges to the prior prevents the detection of regulatory interactions absent from it, a critical drawback that fundamentally constrains model's 's utility for scientific discovery. For example, analyzing gene expression data from cancer cells might reveal a previously unknown transcription factor regulating a pro-oncogenic gene network. Such discovery, which is impossible when confined to a prior network of interactions, could lead to the development of new precision therapies.
To overcome prior-dependent limitations, we propose strategy inspired by learning with privileged information (6. This paradigm obtains a teacher model using supplementary privileged features during training. followed by knowledge transfer to a student model operating without access to such features. Building on this framework, we develop a three-stage architecture named KINDLE sinowledge-guleee Network DistiLlation for frior-free GRN infErence) to infer accurate GRN without relying on prior information. The first stage trains a teacher model integrating both gene expression data and external priors. Notably, inspired by TRIGON's temporal causality modeling 77]. the teacher model explicitly captures temporal regulatory dynamics by predicting future gene expression states from historical expression profiles, rather than relying on static gene co-expression analysis. The incorporated prior knowledge further refines the candidate regulatory space, generating temporally coherent and biologically plausible regulatory maps. The second stage implements knowl- edge distillation to train a student model through teacher supervision while completely eschewing prior information. The final stage deploys the student model for prior-independent GRN inference us- ing expression data exclusively, thereby achieving scalable and unbiased reconstruction of regulatory networks. Our contributions are summarized as follows:
We propose KINDLE to climinate prior dependence in GRN inference by knowledge distillation, which achieves state-of-the-ar performance across four benchmark datasets without requiring prior knowledge.
On mouse embryonic stem cell development data, KINDLE successfully identifies key TFs and predicts their functional roles during differentiation processes.
For mouse hematopoietic stem cell development, KINDLE accurately predicts the effects of Gatal and Spil knockouts on cell fate determination, demonstrating its capability to capture critical regulatory mechanisms.

---

2 Related work
Prior-based GRN inference. Early attempts in GRN inference predominantly relied on co- expression analyses from bulk or single-cell transcriptomic datasets [8H1]] However, the inherent limitations unimodal data approaches became evident due to the vast combinatorial search space of potential TF-TG interactions, which severely restricted their predictive performance. To constrain the solution space, contemporary computational pipelines strategically integrated external biological priors during model optimization. For instance, LINGER (14) employed neural network architecture trained on paired single-cell RNA-seq and ATAC-seq profiles to predict gene expression dynamics through systematic integration of TF abundance and chromatin accessibility. CEFCON [05] imple- mented graph attention network initialized with motif-informer adjacency matrix, synergistically coupling cell lineage-specific GRN inference with network control theory. The Celloracle framework [6] operationalized promoter-enhance interaction maps coupled with DNA motif annotations to establish base GRN architecture, which undergoes iterative refinement through ridge regression. While demonstrating methodological innovation, all these approaches exhibited fundamental depen- dence on the precision and comprehensiveness of incorporated prior knowledge, inaccuracies in prior specification risk propagating systematic biases.
Privileged-feature distillation. Privileged-feaue feature distillation uses auxiliary data accessible ex- clusively during training while eliminating their requirement during downstream deployment. In this paradigm, a teacher model with privileged input creates informative soft targets or latent rep- resentations to supervise a student model restricted to privileged input. Theoretical analyses in learning-to-ran contexts showed that balancing data-driven loss and teacher guidance helps distilled students outperform non-distilled models [6]. Empirically, the BLEND computational framework [7] validated this by applying the methodology to large-scale neurobiological datasets, where be- havioral trajectory data acted privileged supervisory signals during teacher model optimization, and the distilled heural-activitty-nlly student exceled in population coding decryption tasks. Overall, these theoretical and applied advancements established privileged-feature distillation robust way to eliminate the dependence on noisy or resource- inttensve prior information, a critical but underexploited property with potential to enhance GRN inference.
Methodology
3.1 Theoretical Foundation
Our framework is grounded on the causal hypothesis that GRN intrinsically govern transcriptional dynamics through time-evolving interactions. Formally, let G E N denotes the temporal single- cell expression matrix, where N represents temporally ordered cellular states and M is the number of genes. We posit that accurate GRN adjacency matrix A RMx should encode sufficient mechanistic information to predict future expression states from historical observations and thus satisfy:
GT+LT4W F(GITT A)
where F encodes the nonlinear regulatory kinetics, T and W define historical and future time windows respectively. The GRN inference problem is thus reframed as learning a minimal sufficient interaction matrix A that minimizes the difference between predicted and actual gene expression profiles:
A' arg min [[ A) GT+IT+Wll A
We present KINDLE to operationalize this theory and infer the frier-free GRN by three sequential phases: initial supervised training of the teacher model to assimilate prior network guidance, subse- quent distillation of the teacher's 's regulatory insight into lightweight student model, and ultimately deployment of the prior independent student model for high-fidelity GRN inference.
3.2 Teacher Model
As illustrated in Figure 2 the teacher model is equipped with hierarchical attention mechanisms, consisting of temporal and spatial layers. the temporal layer, a lower triangular mask is applied to the attention weights, ensuring each gene's 's expression at time step t only attends to its historical
3

---

Stage Three:
Stage Obtaining Teacher Model
Extracting Gene Regulatory Network
Wodociiii kype
Add Norm
Student Model
Output bayer
Krnou sganogatinnn
Transposs layer
Add Nurn
Mutt--ised
Stage Two: Training Student
bayer
Student Model
Teacher Model
Spatidl laper
Tereporal
Embodding tayer
Entedding layer
All gene aspressionn sion stacked
In Tall Cal
All gene
bilcl BII
expression
Expression dynamics
Expression dynamics gens
gane
Prir Putteck
Figure 2: Illustration of KINDLE framework. The pipeline consists of three stages: (1) Teacher model integrates prior knowledge to learn causal regulatory relationships via explicit modeling of expression state transitions across time windows. (2) With teacher model parameters frozen, knowledge distillation transfers regulatory insights to lightweight student model that operates exclusively on expression data, free of prior inputs. (3) The trained student model is deployed to infer GRN, yielding preou-decoppled network that maintain high accuracy.
states (1. - 1}. This causal constraint mirrors the irreversibility of cellular differentiation, where progenitor cells cannot access transcriptional information from their descendants. The spatial layer employs prior-derived binary mask M E (0, 1 M M. where Mapatial = 1 indicates documented regulatory interaction from gene i to j in prior knowledge. This mask sparsifies attention computation by restricting cross-gene interactions to curated regulatory pairs, effectively pruning unvalidated relationships while preserving interpretability. Architecturally, the temporal layer processes input tensors X E BxT xM (batch size B, time steps T, genes M) and outputs a tensor of identical dimensions. To enable gene-centric regulatory modeling the subsequent spatial layer, we perform axis transposition IR BxxxxM + R Bx xT restructuring the tensor to treat each B
gene's temporal trajectory as an independent sequence. This dimensional reorganization permits parallelized computation of gene-specific attention weights across all M genes while maintaining temporal dependencies. Following spatial attention computation, the tensor undergoes transposition operation B MxT - R T followed by linear projection to R BxW M for W W-step gene expression prediction. The end-to-end framework optimizes regulatory dynamics by minimizing the mean squared error: Lreacher
B
W
M
ΣΣΣ
(-))) t4wm
(0) 2400mmm
B.W. M 6=1 mll=imm
where Ý and Y denote predicted and ground truth expression matrix respectively, indexed by batch 6, forecast window w, and gene m. While the attention matrix A E IR g.M' XM extracted from teacher model's spatial layer during inference could be prior-constrained approximation of the theoretically optimal matrix A' defined in 2 the solution remains fundamentally constrained by its prior- dependent architecture. Specifically, the binary masking operation irreversibly eliminates attention weights for gene pairs absent in the prior knowledge (i.c., positions where M spatiol = 0). thereby restricting the teacher model's attention exclusively
sparse subset of regulatory interactions defined by prior-informed positions (i.e., Mapatial = 1). This prior-induced myopia severely limits applicability to emerging biological systems with incomplete interactome annotations. To overcome this fundamental limitation, we design student model that learns the teacher's 's regulatory knowledge through distillation without inheriting its prior constraints.

---

3.3 Student Model
We formalize the student model as Jag E UVS: G H GT+IT+w), operating exclusively on raw expression matrix G1.T E R RBxIXM without prior network integration. We let Jor be the teacher model and the parameter optimization of the student model aims to minimize the following loss:
I1.e= (Gi.).. Ge.T+um
+1 a)
Σ b
Eaisan (Sos (GL:T) )o.m) ms for 1G M ).mm) )bm
Prediction Loss
Regulatory Distillation Loss
(4)
The hyperparameter (0, governs the trade-off between expression prediction fidelity and regulatory knowledge transfer. Crucially, as diagrammed in Figure 2 the student architecture implements two critical modifications: (1) Elimination of the prior-dependent spatial mask M in attention computation, enabling unrestricted interaction modeling between all gene pairs. (2) Removal of the teacher's temporal layer while preserving temporal causality through distillation, resulting in lightweight model.
Distinct formulations of the distillation loss Lasuill extract different dimensions of the teacher model's 's knowledge. the course this research, we explore four primary distillation strategies within KINDLE framework. Each of these strategies is meticulously designed to convey specific aspects the teacher model's knowledge to the student model, thereby enhancing the latter's s performance and understanding.
Hard Distillation. We optimize this baseline through a predictive congruence objective, where Caismll is constructed direct predictive alignment. Specifically, the framework achieves this by optimizing the squared 1.2-norm divergence between the teacher's terminal predictions and the student's corresponding outputs, enforcing knowledge transfusion via deterministic supervision of final-layer activations:
Edistll (feg (GI.T)o, for (G1.T, M Sppatialy )6,m) =I (GI:)G for (G M spatial
Soft Distillation. This paradigm implements probabilistic knowledge transfer through entropy- regulated distribution matching. The framework introduces temperature parameter 7 to soften the logits before applying the softmax function, formally expressed as:
Cassnn (fag (GiT)) for (G1T M Mapatial )o.m) = KL (( [oog(GiT)amm |0 (Ge rraciial
(
where a is the softmax function, and (-(-Il) is the Kullback-Leillrr divergence.
In addition to the aforementioned hard target distillation and soft probabilistic matching, we develop correlation distillation preserve structural dependencies in feature representations. The core objective is to align the teacher-student correlation manifolds through kernel-induced similarity measures, formalized as:
Cussill (ooo(Grr))baa mm (GI:T, M
)b,m) (C(.f (GI:T)B.m. for (G1.T, M Mspatioly
)bm
(7)
where KC(,, .) denote kernel methods to compute the correlation between output of for and fer To address the challenges posed by the high dimensionality of embedded feature spaces in analyzing complex inter-instance correlations, we propose two different kernel methods effectively capture the high-order correlations between instances within the feature space.
Bilinear Pool. It computes inter-instance correlations through outer product operations. Formally, the Bilinear Pool kernel is defined as:
(ffrr Koilinnar (fes (Gmm))mm for (G1.T, Spptial m, -ffo (Gi:))mm)) (fer (GI.T. M spatial
),m)
Gaussion RBF. This non-linear operator characterizes instance relationships through exponentially decaying similarity metrics, possessing stronger non-linear manifold learning capabilities compared
5

---

Table I: Comparison of the proposed KINDLE framework with other GRN inference methods on four datasets provided by BEELINE 18 Bold values denote the best performance for the corresponding metric.
Methods
mESC
mHSC-R
mHSC-L
mHSCC-GM
AUROC
AUPRC
F1
AUROC
AUPRC
F1
AUROC
AUPRC
FI
AUROC
AUPRC
FI
Withou Prior
GRNBoost2
X
0.537
0.127
0.203
0.397
0.034
0.087
0.515
0.181
0.297
0,474
0.083
0.146
GENIE3 O
0.545
0.137
0.218
0.381
0.042
0.108
0.486
0.183
0.322
0.437
0.078
0.162
Random
0.506
0.083
0.152
0.493
0.087
0.161
0.518
0.135
0.227
0.504
0.083
0.154
CRPCON 15
0.479
0.253
0.429
0.531
0:405
0.551
0.683
0.659
0.675
0.457
0.444
0.647
Prior Guided
Cellornele 16
0.490
0.177
0.305
0.536
0.290
0.420
0.557
0.277
0.368
0,487
0.243
0.401
NetREX 10
0.522
0.128
0.217
0.511
0.117
0.211
0.520
0.177
0.282
0.526
0.144
0.219
Prior Random
0:498
0.318
0.482
0.492
0.389
0.570
0.522
0.551
0.691
0.509
0.461
0.627
KINDLE (Soft distillation)
0.747
0.636
0.519
0.561
0.559
0.691
0.599
0.670
0.752
0.562
0.789
0.864
RINDLE
KINDLE (Hard distillation)
0.753
0.643
0.526
0.564
0.578
0.711
0.599
0.669
0.757
0.569
0,793
0.871
KINDLE (Bilinear Pool)
0.751
0.644
0.521
0.551
0.574
0.723
0.567
0.581
0.761
0.561
0.787
0.867
KINDLE (Gaussian RBF)
0.757
0.646
0.529
0.594
0
0.731
0.600
0,672
0.763
0.570
0.799
0.875
to bilinear methods. The kernel admits low-rank Taylor series approximation while preserving topological structures in feature space. Formally, the Gaussion RBF kernel defined as:
K Kgunssion (fes (G1:))o.m... for (G1r Mapatial) )o.m) exp (- IIfe,
(- l..m fo, (G1.r MM
212
(9)
where À is hyperparameter that controls the width of the gaussian function.
3.3.1 GRN Inference
Given the input gene expression time series G E IR N where N denotes the temporal sequence length and M represents the number of genes, we partition the sequence into segments of length T E N Under the divisibility condition T N, we obtain H = T non-overlapping samples
N
{S()) H each containing T consecutive temporal observations:
((s) = G (T- 1): Ag E (1,.,,
(10)
For each partitioned sample S(s), the student model fog generates attention matrix A (0) e M in its spatial layer. We compute the optimal approximation Å to the theoretical A in Eq through temporal ensemble:
H Σ 9=1
Á H
A
(11)
The final GRN Spred is established by ranking the edge weights in the matrix À and selecting the top-k most significant connections, where & corresponds exactly to the number of edges in the ground truth regulatory network Gyt provided with each benchmarking dataset:
Spred = {(,,)) Ai E (á)}. k == 1
(12)
The detailed pseudocode implementations of KINDLE are provided in Appendix
4
Experiments
4.1 KINDLE Achieved State-of-the-Art Performance in GRN Benchmarks
The evaluation of KINDLE strictly adheres to the benchmarking protocol introduced in BEELINE [18. We systematically validated our approach on four mouse diffcrentiation datasets, the embry- onic stem cell (mESC) dataset as well as three hematopoietic lincages: Erythrocyte (mHSC-E), Sauuulocyte-Monccyye
(mHSC-GM), and Lymphocyte (mHSC-L). Following BEELINE's estab- lished framework, we treated GRN inference as binary classification task, employing lineage- matched reference GRN derived from ChIP-seq experiments as ground truth (see Appendix Ior detailed information). Performance was quantified through three metrics: area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and F1 score (see Appendix [B.2 for pseudocode of metric calculation). KINDLE was compared against seven competitiee baselines: sxpression-basee methods (GENIE3 99. GRNBoost2 (8). brior-based

---

Stage
Stage
Stage
Stage
Stage
Nanog
Gata4
Kdmšo
965
Sox2
Nanog
-349
Ulft
Zfp42
4
Esrrb
Trp63
Sap30
004
Nr0b1
Nya-
Teft2
sove
-002
Akit
Soxt7
Mybiz
EIM1 Trim28
Eas
Egrt
Ponl
Trim2
Jarid2
Z-scored AUC score based on inferred GRN
Figure Temporal dynamics of TF regulatory performance during mouse embryonic cell differentiation. Left: Heatmap visualization of z-scored AUC scores reveals bimodal temporal pat- terns through hierarchical clustering. Two distinet TF clusters emerge, demonstrating their divergent regulatory roles during differentiation. Right: Trend analysis of AUC score, with developmental time points on x-axis and scored AUC score on y-axis. Red curves represent quadratic polynomial fits to the dynamic profiles. Complete regression curves for all 25 TFs are available in Appendix
approaches (CEFCON [5], CellOracle [16], NetREX [9] ) and two random controls (Random, Prior_ Rood___nndmm.... Detailed descriptions of datasets, ground truth networks, and baseline models are provided in Appendix A.I Appendix A.2 and Appendix A.3 respectively.
As shown Table T all four KINDLE variants demonstrated substantial improvements over baselines despite requiring external biological priors. The soft distillation variant, representing our weakest configuration, surpassed the best expression- based method (GENIE3) by 0.499, 0.517, 0,490, 0.711 in AUPRC across datasets and outperformed CellOracle by 0.546 in mHSC-GM. Among variants, the one with Gaussian RBF (hereafter KINDLE-Gaussial achieved the best performance in 11 of 12 matric combinations. On the mESC dataset, KINDLE- E-auassiiiin improved AUROC from 0.545 (GENIE3) to 0.757 (39% increase), elevated AUPRC from 0.253 (CEFCON) to 0.646 (155% improvement). and raised F1 score from 0.429 to 0.529 (23% gain). Comparable enhancements emerged in hematopoietic lineages: AUPRC increased by 48% (0.405 + 0.601) for erythroid differ- entiation and nearly doubled (0.444 - 0.799) granulocyte-monocye development, accompanied by 0.228 absolute Fl score improvement.
Notably, KINDLE's superiority proved most pronounced in AUPRC and Fl metrics. As summa- rized in Table 2 the validated edges in ground truth network constituting merely 0.9% (mESC), 0.65% (mHSC-E), 0.7% (mHSC-GM), and 1.15% (mHSC-L) of candidate edges, thus introducing severe class imbalance in the binary classification task. Under such conditions, AUPRC and F1 serve as more reliable performance indicators than AUROC (detailed justifications are provided in Appendix B.3). Therefore, the consistent AUPRC and F1 improvements demonstrated that privileged knowledge distillation provided robust, prior-free route GRN inference, outperforming not only expression-only algorithms but also methods that rely on explicit biological priors.
4.2 KINDLE Identified Key TF' and Their Stage-Specific Functions
Beyond quantitative metrics for assessing GRN accuracy, crucial evaluation criterion lies in determining whether the inferred GRN can identify key TF' that orchestrate differentiation processes. SCENIC 2 introduced the AUCell algorithm, which calculates AUC scores for TF regulon (the set of target genes TF the inferred GRN) through rank-based enrichment analysis. This score reflects the functional activity of the TF regulon within each cell, enabling systematic identification

---

(a)
(c)
Pseudotime for each cell
Gata1 expression
Perturbation vector of Gata1 knockout
Gonsunnppeed & Venscaa inaage (1NSS-001
Brffroogtts lmaag jei BC-5)
sxpression High
Erythrocytos linaago
CMP
Lynono ineegg
(1H604)
Gone expression
(b)
(d)
Differentiation vector
Spitt expression
Perturbation vector of Spi1 knockout
CMP
ia
Bryhoggfew Ineage
GMP
LMPP
GMP
CMP
MEP
Figure 4: In silico perturbation analysis validates that KINDLE can accurately predict the cell fate transitions during the multi- Mineie haematopoietic differentiation. (A) Slingshot-derived pseudotime trajectory embedding. (B) Cell-type annotations overlaid on the pseudotime landscape. (C) Left: Expression profiles of Gatal. Right: Silencing Gatal induces reverse perturbation vectors along the erythroid branch and leads to differentiation arrest in CMP and MEP. (D) Left: Expression profiles of Spil. Right: Silencing Spil represses the differention of LMPP and GMP and promotes erythroid progression in CMP.
of key regulators that drive cellular state transitions (see Appendix Coror detailed methodology of AUCelI). Following this protocol, we computed AUC scores from the GRN inferred by KINDLE- Gaussian to estimate per-cell TF activities in the mESC dataset. Given the five differentiation stages in this dataset (sce Appendix A..]). we performed analysis of variance on AUC scores to assess whether they varied significantly throughout the differentiation stages and defined key TFs as those with Hochiiii--oheee---aeee
P-values 0.01. We identified 25 key TFs, with their adjusted P-values listed in Table 3 Notably, 18 (72%) of 25 identified regulators have established roles in mESC differentiation according to prior literature 21
Temporal patterning of the 25 TF regulon activities was investigated through hierarchical clustering of z-scored AUC scores (Figure 3). Two anti-correlated activation modules emerged from this unsupervised analysis: Early-stage regulators (Nanog, Sox2, NrOb1, etc.) demonstrated peak activity stage one with progressive attenuation through subsequent stages. This temporal trend aligns with known biological functions, such as Nanog and Sox2 being highly expressed in the early stage of mouse embryonic stem cells, maintaining the pluripotency of stem cells [42]. Their expression rapidly decreases as cells commit to differentiation, reflecting their pivotal function in regulating the transition from a pluripotent state more specialized lineages (44 Late-stage regulators (Gata4, Sox17, Kdm5b, etc.) exhibited minimal initial activity but showed significant activation from stage three onward. These results corroborate established mechanisms of lineage specification, where 1ox overexpression upregulates a set of aodooder--spccinn gene markers and induces an ESC differentiation program towards primitive endoderm 4k] The emergence of these antiphasic expression patterns demonstrated that KINDLE not only recovered biologically relevant TFs but also assigned each regulator to its stage-specific functional context, thereby elucidating the sequential deployment of transcriptional programs during mESC differentiation.
8

---

4.3 KINDLE Predicted Lineage-Specific Fate Changes in In-Silico Perturbation
Following the precise identification of key TFs, a practical application involves interrogating their functional roles through systematic perturbation. We employed the Celloracle framework [16] to implement in silico perturbation analysis. This approach simulates TF knockout by setting target TF expression to zero and propagating the perturbation signal through the GRN's topological structure to its target genes, ultimately generating two-dimensional perturbation vector for each cell that predicts its fate trajectory under the specified perturbation (see Appendix [ for algorithmic details). To validate the biological relevance of KINDLE -aussian'' 's predictions, we applied this methodology to the mHSC dataset, an ideal and complex benchmark system containing six distinct cell types (HSC, MPP, LMPP, GMP, CMP, MEP; see Appendix A.I for cell type information) organized along three differentiation trajectories (Figure 4 a.b). The sequential differentiation order of different cell types is shown in Figure
We focused on two well-characterized regulators governing hematopoietic lineage commitment, Gatal and Spi Consistent with their established roles, Gatal expression dominated in erythroid-lineage cells (CMP and MEP, Figure{p), while Spil showed myeloid-lineage enrichment (LMPP and GMP, Figure I). Following the perturbation of Gatal, we generated perturbation vectors for each cell. Notably, the vectors for all cells within the erythroid lineage were opposite to the developmental direction of pseudotime trajectory shown in Figure 4a.. This observation indicated that in the absence of Gatal, cells tend to revert to earlier progenitor states rather than progress towards more mature cell identities. To quantitatively assess the perturbation effect, we calculated a perturbation score for each cell and coloured the cells (purple - negative score, differentiation inhibited; green - positive score, differentiation promoted, see Appendix D.3e for additional details of perturbation score calculation). In erythroid lineage, all cells received negative scores, with the strongest inhibitory effect concentrated in CMP and MEP, the cell populations with the highest Gatal expression levels. Subsequently, we applied the same perturbation procedure Spil. Upon silencing Spil, all cells showed a developmental trajectory towards the erythroid lineage, with CMP differentiation being promoted as well GMP and LMPP differentiation being inhibited (Figure ft). These perturbation results are consistent with previous reports 444, where Gatal promotes the differentiation of CMP into MEP (resulting in inhibitation of CMP and MEP differentiation when Gatal knockout), and Spil suppresses the CMP to MEP transition (resulting in CMP perturbation vectors pointing towards MEP upon Spil silencing).
Collectively, the in silico perturbation analyses demonstrated that within the haematopoictic system, KINDLE accurately modeled the downstream effects of key TFs knockout. Hence, beyond pinpointing key TFs, KINDLE provided a mechanistic scaffold for the rational design of cell-fate-engineere strategies.
4.4 Implementation Details
We trained KINDLE on an 80GB Nvidia A100 GPU for 30 epochs with batch size of 32. To prevent overfitting, we implemented an early stopping strategy with a patience value set to 3. For optimization, we employed the Adam optimizer conjunction with a warmup strategy, gradually increasing the learning rate from 0 to le-4. Subsequently, a CosineAnnealingL.. scheduler was utilized to further fine-tune the learning rate. During the training process, we conducted experiments with five distinct values for the hyperparameter W (1, 2, 4, 8, and 16), ultimately selecting W = 16 as it yielded the optimal results reported in this paper.
5 Discussion and Limitations
KINDLE advances GRN inference methodology by decoupling algorithm from prior knowledge dependency (the longstanding bottleneck in the field). Through integrating temporal causality modeling with knowledge distillation, our framework successfully transfers regulatory insights learned from privileged prior- rrgge-ed data frior-free student model, enabling KINDLE to achieve state-of-the thet----tt performance across four benchmark datasets. The model's 's ability to recover key TFs governing lineage specification validates its capacity to capture biologically interactions and the accurate prediction of knockout effects on hematopoietic cell fate transition underscores its potential for elucidating dynamic regulatory mechanisms in development and disease. The framework's 's prior-independent nature positions it as versatile tool for studying poorly characterized

---

systems, such as non-model organisms emerging pathological states, where reliable prior networks are often unavailable.
Despite its strengths, KINDLE has several limitations. First, its reliance on temporal gene expression data restricts applicability datasets with longitudinal sampling. Second, the distillation process may inherit biases from the teacher model's prior-dependent training phase, potentially propagating errors from incomplete or noisy priors. Third, the current implementation focuses transcriptional regulation, omitting post-transeriptional and epigenetic layers of gene regulation that could refine network predictions. Addressing these challenges will be critical for extending the framework utility across diverse biological contexts.
Acknowledgments and Disclosure of Funding
This research was supported by National Key Research and Development Program of China (2044YFF0007400) and National Natural Science Foundation of China (6220071694).
References
[1] Pau Badia-i Mompel, Lorna Wessels, Sophia Müller-Dott, Rémi Trimbour, Ricardo O Ramirez Flores, Ricard Argelaguet, and Julio Saez-Rodriguez. Gene regulatory network inference in the era of single-cell multi-omics. Nature Reviews Geneties, 24(11):739-755 2023.
[2] Albert-Laszlo Barabasi and Zoltan Oltvai. Network biology: understanding the cell's functional organization. Nature reviews genetics, 5(2):101-113, 2004.
[3] Guy Karlebach and Ron Shamir. Modelling and analysis of gene regulatory networks. Nature reviews Molecular cell biology, 910)) 2008.
[4] Jason D Buenrostro, Beijing Wu, Ulrike M Litzenburger, Dave Ruff, Michael L Gonzales, Michael P Snyder, Howard Y Chang, and William Greenleaf. Single-cell chromatin accessi- bility reveals principles of regulatory variation. Nature, 523(7561):486 490, 2015.
[5] Erez Lieberman-Aides Nynke L Van Berkum, Louise Williams, Maxim Imakaev, Tobias Ragoczy, Agnes Telling, Ido Amit, Bryan R Lajoie, Peter J Sabo, Michael O Dorschner, et al. Comprehensive mapping of long-range interactions reveals folding principles of the human genome. science, (26(5950):289-299) 2009.
[6] Shuo Yang, Sujay Sanghavi, Holakou Rahmanian, Jan Bakus, and Vishwanathan SVN. To- ward understanding privileged features distillation in learning-to-rank, Advances in Neural Information Processing Systems, 15:26658-226777 2022.
[7] Rui Peng. Juntian Qi, Yuxing Lu, Wei Wu, Qichen Sun, Chi Zhang, Yihan Chen, and Jinzhuo Wang. Dissecting dynamic gene regulatory network using transformer-based temporal causality analysis. bioRxiv, pages 2025-02, 2025.
[8] Thomas Moerman, Sara Aibar Santos, Carmen Bravo González-Bie Blas, Jaak Simm, Yves Moreau, Jan Aerts, and Stein Aerts. Grnboost2 and arboreto: efficient and scalable inference of gene regulatory networks. Bioinformatics,
2019.
[9] Vân Anh Huynh-Thu, Alexandre Irrthum, Louis Wehenkel, and Pierre Geurts. Inferring regulatory networks from expression data using tree- breebased methods. PloS one, 5(9):e1277.. 2010.
[10] Alicia T Specht and Jun Li. Leap: constructing gene co-expression networks for single-cell ma- massequencing data using pseudotime ordering. Bioinformatics, 33(5):764-766, 2017.
[11] Hirotaka Matsumoto, Hisanori Kiryu, Chikara Furusawa, Minoru SH Ko, Shigeru BH Ko, Norio Gouda, Tetsutaro Hayashi, and Itoshi Nikaido. Scode: efficient regulatory network inference algorithm from single-cell rna-seq during differentiation. Bioinformatics, 33(1))33333333 2017.
10

---

[12] Thalia E Chan, Michael PH Stumpf, and Ann C Babtie. Gene regulatory network inference from single-cell data using multivariate information measures. Cell systems, 5(3):251 2017.
[13] Shahin Mohammadi, Jose Davila- Velderrain, and Manolis Kellis. Reconstruction cell-type- specific interactomes single-cell resolution. Cell systems, 9(6):559-505 2019.
[14] Qiuyue Yuan and Zhana Duren. Inferring gene regulatory networks from single-cell multiome data using atlas-scale external data. Nature Biotechnology, pages 1-11, 2024.
[15] Peizhuo Wang, Xiao Wen, Han Li, Peng Lang, Shuya Li, Yipin Lei, Hantao Shu, Lin Gao, Dan Zhao, and Jianyang Zeng. Deciphering driver regulators of cell fate decisions from single-cell transcriptomics data with cefcon. Nature Communications, 14(1):8459, 2023.
[16] Kenji Kamimoto, Blerta Stringa, Christy M Hoffmann, Kunal Jindal, Lilianna Jalnica-Krezed and Samantha Morris. Dissecting cell identity via network inference and in silico gene perturbation. Nature, 614(7949),742-751 2023.
[17] Zhengrui Guo, Fangxu Zhou, Wei Wu, Qichen Sun, Lishuang Feng, Jinzhuo Wang, and Hao Chen. BLEND: Behavior-guided neural population dynamics modeling via privileged knowledge distillation. In The Thirteenth International Conference on Learning Representations 2025.
[18] Aditya Pratapa, Amogh P Jalihal, Jeffrey N Law, Aditya Bharadwaj, and TM Murali. Bench- marking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nature methods, 17(2):147-154, 2020.
[19] Yijie Wang, Dong- Yeon Cho, Hangnoh Lee, Justin Fear, Brian Oliver, and Teresa M Przytycka. Reprogramming of regulatory network using expression uncovers sex-specific gene regulation drosophila. Nature communications, 9(1 9(1):4061, 2018.
[20] Sara Aibar, Carmen Bravo González-Blas, Thomas Moerman, Vân Anh Huynh-Thu, Hana Imrichova, Gert Hulselmans, Florian Rambow, Jean-Chrisee ChrisChiste Marine, Pierre Geurts, Jan Aerts, et al. Scenie: single-cell regulatory network inference and clustering. Nature methods, 14(11):1083-14 1086, 2017.
[21] Claire Soudais, Malgorzata Bielinska, Markku Heikinheimo, Craig A MacArthur, Naoko Narita, Jeffrey E Saffitz, M Celeste Simon, Jeffrey M Leiden, and David B Wilson. Targeted mutagenesis of the transcription factor gata-4 gene in mouse embryonic stem cells disrupts visceral endoderm differentiation in vitro. Development, 121(1 1:37777-3888, 1995.
[22] Benjamin L Kidder, Gangqing Hu, and Keji Zhao Kdm5b focuses h3k4 methylation near promoters and enhancers during embryonic stem cell self-renewal and differentiation. Genome biology, 15:1 1-9, 2014.
[23] Janel Kopp. Briana D Ormsbee, Michelle Desler, and Angie Rizzino. Small increases in the level of sox2 trigger the differentiation of mouse embryonic stem cells. Stem cells, 26(4):903-911, 2008
[24] Yuin-Han Loh, Qiang Wu, Joon-Lin Chew, Vinsensius B Vega, Weiwei Zhang, Xi Chen, Guillaume Bourque, Joshy George, Bernard Leong, Jun Liu, et al. The oct4 and nanog transcription network regulates pluripotency in mouse embryonic stem cells. Nature genetics, (8(4):431-440 2006.
[25] Guangjin Pan and James A Thomson. Nanog and transcriptional networks in embryonic stem cell pluripotency. Cell research, 17(1):42-9, 2007.
[26] Akihiko Okuda, Akiko Fukushima, Masazumi Nishimoto, Akira Orimo, Toshiyuki Yamagishi, Yoko Nabeshima, Makoto Kuro-o, Yo-ichi Nabeshima, Kathy Boon, Marie Keaveney, et al. Utfl, a novel transcriptional coactivator expressed pluripotent embryonic stem cells and extra-cmbryonic cells. The EMBO journal, 1998.

---

[27] Fariba Dehghanian, Patrick Piero Bovio, Fabian Gather, Simone Probst, Amirhosein Naghsh- Nilchi, and Tanja Vogel. Zíp982 confers mouse embryonic stem cell characteristics by regulating expression of nanog, zfp42, and dppa3. Biochimica el Biophysica Acta (BBA)-Molecular Cell Research, 1871 116666, 2024.
[28] Eiichi Okamura, Oliver H Tam, Eszter Posfai, Lingyu Li, Katie Cockburn, Cheryl QE Lee, Jodi Garner, and Janet Rossant. Esrrb function is required for proper primordial germ cell development in presomite stage mouse embryos. Developmental biology, 455(2):382-392, 2019.
[29] Xiaofei Zhang, Juan Zhang, Tao Wang, Miguel A Esteban, and Duanqing Pei. Esrrb activates oct4 transcription and sustains self-renewal and pluripotency in embryonic stem cells. Journal of Biological Chemistry, 283(51)
2008
[30] Tianji Chen, Juan Du, and Guangxiu Lu. Cell growth arrest and apoptosis induced by oct4 or nanog knockdown in mouse embryonic stem cells: a possible role of trp53, Molecular biology reports, 39: 18511855-188 1861, 2012.
[31] Setsuko Fujii, Satomi Nishikawa- Torika,,, Yoko Futatsugi, Yayoi Toyooka, Mariko Yamane, Satoshi Ohtsuka, and Hitoshi Niwa. NrObl is a negative regulator of zscan4c in mouse embryonic stem cells. Scientific reports, 5(1):9146, 2015.
[32] Diego Pasini, Adrian P Bracken, Jacob B Hansen, Manuela Capillo, and Kristian Helin. The polycomb group protein suz12 is required for embryonic stem cell differentiation. Molecular and cellular biology, 2007.
[33] Diego Pasini, Adrian P Bracken, Michael R Jensen, Eros Lazzerini Denchi, and Kristian Helin. Suz12 is essential for mouse development and for ezh2 histone methyltransferase activity. The EMBO journal, 23(20):4061 4061-44166 2004.
[34] Kathy K Niakan, Hongkai Ji, René Maehr, Steven A Vokes, Kit T Rodolfa, Richard I Sherwood, Mariko Yamaki, John Dimos, Alice Chen, Douglas Melton, et al. 17 promotes differentiation in mouse embryonic stem cells by directly regulating extraembryonic gene expression and indirectly antagonizing self-renewal. Genes & development, 24(3)311-3266 2010.
[35] Ismail Kola, Sarah Brookes, Anthony R Green, Richard Garber, Martin Tymms, Takis S Papas, and Arun Seth. The ets1 transcription factor is widely expressed during murine embryo development and is associated with mesodermal cells involved in morphogenetic processes such as organ formation. Proceedings of the National Academy of Sciences, 90(16):7588-7592 1993.
[36] Sanjay K Singh, Mohamedi N Kagalwala, Jan Parker-Thornbure Henry Adams, and Sadhan Majumder. Rest maintains self-renewal and pluripotency of embryonic stem cells. Nature, 453(7192):223-227, 2008
[37] Anna Poteé, Alexander Rowe, Pernille Blicher, Rajikala Suganthan, Magnar Bjørås, and Stig Ove Bøe. Pml regulates the epidermal differentiation complex and skin morphogenesis during mouse embryogenesis. Genes, 1(10):1130, 2020.
[38] Yaser Atlasi, Rubina Noori, Claudia Gaspar, Patrick Franken, Andrea Sacchetti, Haleh Rafati, Tokameh Mahmoudi, Charles Decraene, George A Calin, Bradley J Merrill, et al. Wnt signaling regulates the lineage differentiation potential of mouse embryonic stem cells through tef3 down-regulation. PLoS genetics, (5) 2013.
[39] Helen M Rowe, Adamandia Kapopoulou. Andrea Corsinotti, Liana Fasching, Todd Macfarlan, Yara Tarabay, Stéphane Viville, Johan Jakobsson, Samuel L Pfaff, and Didier Trono. Trim28 repression of retrotransposon-beee based enhancers necessary to preserve transcriptional dynamics in embryonic stem cells. Genome research, 23(3):452-461, 2013.
[40] Eunjin Cho, Matthew R Mysliwiec, Clayton D Carlson, Aseem Ansari, Robert J Schwartz, and Youngsook Lee. Cardiac-specifie developmental and epigenetic functions of jarid2 during embryonic development. Journal of Biological Chemistry, 293(30): 2018.
12

---

[41] Songhwa Kang, Jisoo Yun, Scok Yun Jung, Yeon Ju Kim, Ji Hye Park, Seung Taek Ji, Woong Bi Jang, Jongseong Ha. Jae Ho Kim, Sang Hong Back, al. Adequate concentration of b cell leukemia/lymphona 3 (bel3) is required for pluripotency and self-renewal of mouse embryonic stem cells via downregulation of nanog transcription. BMB reports, 51 (2):92, 2018
[42] Shinji Masui, Yuhki Nakatake, Yayoi Toyooka, Daisuke Shimosato, Rika Yagi, Kazue Takahashi, Hitoshi Okochi, Akihiko Okuda, Ryo Matoba, Alexei A Sharov, et al. Pluripotency governed by sox2 via regulation of oct3/4 expression in mouse embryonic stem cells. Nature cell biology, 96)):625663,, 2007.
[43] Xue-Bin Qu, Jie Pan, Cong Zhang, and Shu- Yang Huang. Sox17 facilitates the differentiation of mouse embryonic stem cells into primitive and definitive endoderm in vitro. Development, growth differentiation, 50(7):585 2008.
[44] Kinuko Ohneda and Masayuki Yamamoto. Roles of hematopoietic transcription factors gata-1 I and gata-2 in the development of red blood cell lineage. Acta haematologica, 108(4):237-245, 2002.
[45] Yuko Fujiwara, Carol P Browne, Kerrianne Cunniff, Sabra C Goff, and Stuart H Orkin, Arrested development of embryonic red cell precursors mouse embryos lacking transcription factor gata-1. Proceedings of the National Academy of Sciences, 93(22):
1996.
[46] Laura Gutiérrez, Noemí Caballero, Luis FernándezCaalleja. Elena Karkoulia, and John Strouboulis. Regulation gatal levels in erythropoiesis. JUBMB life, 72(1))89-105, 2020.
[47] Pu Zhang, Xiaobo Zhang, Atsushi Iwama, Channing Yu, Kent A Smith, Beatrice U Mueller, Salaija Narravula, Bruce E Torbett, Stuart H Orkin, and Daniel G Tenen. Pu. inhibits gata-1 1 function and erythroid differentiation by blocking gata-l dna binding. Blood, The Journal of the American Society of Hematology, 96(8):2641-2655 2000.
[48] Kelly Street, Davide Risso, Russell B Fletcher, Diya Das, John Ngai, Nir Yosef, Elizabeth Purdom, and Sandrine Dudoit. Slingshot: cell lineage and pseudotime inference for single-cell transcriptomics. BMC genomics, 19:1-16, 2018
[49] Ronald R Coifman, Stephane Lafon, Ann B Lee, Mauro Maggioni, Boaz Nadler, Frederick Warner, and Steven W Zucker. Geometric diffusions tool for harmonic analysis and structure definition of data: Diffusion maps. Proceedings of the national academy of sciences, 102(21):7426-4411 2005.
[50] ENCODE Project Consortium et al. An integrated encyclopedia of dna elements in the human genome. Nature, 489(7414):57, 2012.
[51] Zhaonan Zou, Tazro Ohta, and Shinya Oki. Chip-atlas 3.0: data-mining suite to explore chromosome architecture together with large-scale regulome data. Nucleic Acids Research, 52(W1)W45-W53, 2024.
[52] Huilei Xu, Caroline Baroukh, Ruth Dannenfelser, Edward Y Chen, Christopher M Tan, Yan Kou, Yujin E Kim, Ihor R Lemischka, and Avi Ma' ayan. Escape: database for integrating high-content published data collected from human and mouse embryonic stem cells. Database, 20 2013:bat045, 2013.
[53] Luz Garcia-Ale Alonso, Christian H Holland, Mahmoud M Ibrahim, Denes Turei, and Julio Saez- Rodriguez. Benchmark and integration of resources for the estimation of human transcription factor activities. Genome research, 29(8):1363-1375 2019.
[54] Zhi-Ping Liu, Canglin Wu, Hongyu Miao, and Hulin Wu. Regnetwork: integrated database of transcriptional and post-transeriptional regulatory networks in human and mouse. Database, 2015:bav095, 2015.
[55] Heonjong Han, Jae- Won Cho, Sangyoung Lee, Ayoung Yun, Hyojin Kim, Dasom Bae, Sunmo Yang, Chan Yeong Kim, Muyoung Lee, Eunbeen Kim, et al. Trrust v2: an expanded reference database of human and mouse transcriptional regulatory interactions. Nucleic acids research, (3100):330-DD380 2018
13

---

[56] Damian Szklarezyk, Annika L Gable. David Lyon, Alexander Junge, Stefan Wyder, Jaime Huerta- Huerra-Cepas. Milan Simonovic, Nadezhda T Doncheva, John Morris, Peer Bork, al. String v11: protein-protent association networks with increased coverage, supporting functionnl discovery in genome- widd experimental datasets. Nucleic acids research, 47(D1)):607-DD13, 2019.
[57] Jesse Davis and Mark Goadrich. The relationship between precision-recall and curves. In Proceedings of the 23rd international conference on Machine learning, pages 233-240, 2006.
14

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer:
[Yes]
Justification: We provide experimental evidence which supports our claims and contributions in Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

This article is application oriented and develops an algorithm to address the practical problems existing in the field of gene regulatory networks, rather than being a theoretical derivation article.
Justification:
sipiication-orieneed
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provided comprehensive details regarding all datasets and baseline methods in Appendix [A] Additionally, Section (e.cioon[.4]elateee elaborates on the hardware infrastructure employed during training. along with specifics such as optimizer configuration, batch size selection, and other relevant hyperparameters.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
16

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: All datasets used in this study are publicly available, and detailed descriptions can be found in Appendix [A.1 Additionally, the code implementation of KINDLE is thoroughly documented in Appendix F
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-sourge benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
ctspps.ccppulli//uuieee gwwies/ CodeeSumiiiseeeeee onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provided comprehensive details about the hardware infrastructure employed during training, along with specifics such as optimizer configuration, batch size selection, and other relevant hyperparameters Section 4.4
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Table 3 shows the P-values of the 25 transcription factors identified by KIN- DLE.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We elaborated on the compute resources during our training process in Section
4.4
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdictio).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
we discussed the advantages and disadvantages of KINDLE, as well its practical value and impact in areas lacking prior knowledge.
Justification: In Section 5
18

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do released data or models that have high risk for misuse.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual- should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cited all the papers that are relevant to the code, models, and datasets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
19

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[Yes]
Justification: We provided the full set of pseudocode for the implementation of KINDLE in Appendix FF
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question:
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This article does involve any human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This article does not involve any human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Large language models are not important, original, or non-standard components of the core methods in this study.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurl)) hutt:e///eerrrppccc///offeereeee /tt///////oeeeeeee////////../. for what should or should not be described.
21

---

A Supplementary Contents of Datasets and Baselines
A.1 Datasets
Mouse embryonic stem cell (mESC). The mESC dataset contains Single-cell RNA Granulocytes
Monocytes
Erythrocytes
quencing (scRNA-seq) expression measure- ments for 421 primitive endoderm cells differen- tiated from mESCs, collected at five time points (0, 12, 24, 48, and hours). Pseudotime com- putation was performed using Slingshot [48]. with cells at 0 hours as the starting cluster and cells at 72 hours as the terminal differentiation state.
MEP
GMP
Lymphoid
CMP
LMPP
MPP
Mouse hematopoietic stem cell (mHSC). The mHSC dataset comprises 1,656 hematopoi- etic stem and progenitor cells traversing six critical differentiation states: hematopoietic stem cells (HSCs), multipotent progenitors (MPPs), hymphoid-primnn -rrimedd multipotent progen- itors (LMPPs), common myeloid progenitors (CMPs), gggakaryooct--erythrooyy. progeni- tors (MEPs), and granulocyte mooocyt progenitors (GMPs). As visualized in Figure these cell types follow distinct differentiation trajectories across three developmental lineages. Pseudotime trajectories were computed using the first three principal dimensions derived from DiffusionMMp
HSC
Figure 5:
Differentiation schematic of six cell states in the mHSC dataset.
[49]
Gene regulatory networks were independently reconstructed for each lineage.
A.2 Ground Truth Networks
To benchmark inferred GRN, BEELINE 18 constructed ground truth networks from three stratified categories:
Cell-type-ppecifii networks:
Sourced from ENCODE [50), ChIP-Atlas (51] and ESCAPE 52 databases
Matched to the scRNA-seq dataset's cell lineage
Included loss-of-fumction or gain-of-function perturbation data from ESCAPE
Non-specific networks:
DoRothEA 553 Integrated regulatory interactions filtered by confidence levels:
Level A: high-confidence ChIP-seq data
* Leveel B: likely-confidence interactions
RegNetwork 54 Genome-wide TF- gene and TF-TF interactions across human and
mouse
-RRUST 55
Manually curated TF-TG pairs from literature mining
Functional networks:
Derived from STRING [56 protein interaction databases
Captured indirect regulatory effects (e.g., phosphorylation, co-expression)
Notably, our study, cell-type-specific networks were employed as the ground truth for the bench- mark evaluation of our model.
A.3 Baselines
GENIE3 decomposes GRN inference into p regression problems for p genes, using tree-based ensembles to quantify regulatory potential. For each target gene, it evaluates the predictive importance of all other genes' expression patterns as putative regulators. These pairwise importance scores aggregated to rank regulatory interactions and reconstruct directed networks.
GENIE3 9
22

---

GRNBoost2 [8]. GRNBoost2 is gradient-boosting boosting-based based algorithm for GRN inference. Inspired by GENIE3, it trains tree-based regression models predict each gene's expression profile using TF expression data. The method employs regularized stochastic gradient boosting with an early-stopping heuristic: training terminates when out-of-bag data indicates non-improving loss function (average improvement < 0). Regulatory associations are aggregated and ranked by importance scores to construct the final GRN.
CEFCON [3].. CEFCON is network control theory framework for cell fate analysis using scRNA- seq data. It constructs lineage-specific GRN via graph attention neural network under contrastive learning, aggregating gene interactions through adaptive neighborhood weighting. By integrating minimum feedback vertex sets and minimum dominating sets with GRN influence scores, it identifies driver regulators of cell fate transitions.
CellOracle [66.. CellOracle integrates ScATAC-seq motif analysis and scRNA-seq data to model context-dependent GRNs. It first builds a base network through TF-binding motif scanning of regulatory DNA, and then refines edge weights using regularized linear models expression data. Through silico TF perturbation simulations, it predicts cell identity shifts by propagating signals across the GRN and analyzing pscudotime gradient vector fields.
NetREX reconstructs context-specific GRN by optimizing prior networks against expression data. Formulated non-convex lo-norm optimization problem, iteratively modifies network topology using proximal alternative linearized maximization.
NetREX 19].
Random. The Random baseline generates GRN by randomly selecting edges from all possible gene-gene interactions, where l equals the number of edges in the ground truth GRN.
Prior_Randol Random. Rooo_Randommma selects edges exclusively from prior network interactions (k matches the number of ground truth GRN edges).
B Supplementary Contents of Benchmark Testing
B.1
The evaluation of GRN is regarded as binary classification problem
The evaluation of inferred GRN can be formal- ized as a binary classification task, where edges in the inferred network are categorized relative to ground truth GRN. depicted the Fig- ure 6, let Gime (V. Firue) denote the ground truth network, and Gpeed = (V, Fepres) represent the inferred network. Each edge E Eped is classified into one four categories: (1) True Positive (TP) E Exree n Eirue. (2) False Pos- itive (FP): : E Epred Euree- (3) True Negative (TN) e Epred Euue- (4) False Negative (FN) e E Eure Epred- Performance metrics are de-
GRN
Inferee
Target
Real
Predicted edge
Predicted true postivv edge
Predicted true negative edge
Predicted taise posilvv
Predicted taise
Figure 6: Schematic diagram of GRN evaluation, where the predicted edges can be classified into four types.
rived as follows: Precision = TP+Fp- Recall = 2
TP
TP
F1
Precision Recall
TPR TP
TP+
Precision + Recal]
TP#FN
FPR FP4TN AUROC J/¹ d AUPRC = fe Precision d Recall. These metrics enable
FP
systematic comparison of GRN inference methods, quantifying both accuracy and robustness.
23

---

B.2 GRN Benchmakring Algorithm
Algorithm GRN Benchmark Evaluation
1: Input:
2: Ground truth GRN: Gtme = (V, Emme)
3: Predicted GRN for N algorithms: { pred (V, Fre |
4: Output: Metries R x3
DataFrame containing AUROC, AUPRC, F1
5: // Preprocess ground truth
6: Fine f- Firae - {(v, v) E
Remove self-loops
7: Farue +- Deduplicate Pirue)
Remove duplicate edges
8: results +-
> Initialize metric collection
9: for each predicted GRN Gpedd do
10: E(() f- Sor((edduplicate( prea)) prea))
Sort predicted edges
Generate candidate edges
12:
Pan < V V {(e, v)
D All potential non-self edges Obtain true labels, where is the indicator function
Virue +- [Te E Finue) Ve Pan)
14:
I/ Obtain predicted edges
y + TopK (5()) pred)
Select the top edges
Calculate metries
17:
TPR (1) FPR (3)
TP
FP
> ROC components
FP
FP4TN
18:
Precision" (9) Recall
TP
PR components
TP TP+FN
19:
F1 (9) +- 2 Precision (9) (ecall
D Calculate F1 score
Precision e1 +R
20:
AUROC (UPRC()) + TPR fo Precision d Recall
results +- results AUROC() (UPRC()) F]
b Record metrics
22: end for
// Aggregate results
24: Metrics <- ConstruciDataFrameerrsss
Shape: N x
25: return Metrics
B.3 AUPRC is more robust compared to AUROC
Table 2: The distribution of positive and negative labels in four datasets
mESC
mHSC-E
mHSC-GM
mHSC-L
Genes
1652
1933
1520
640
Potential edges
2727452
3734556
230880
408960
True edges
24557
24726
16198
4705
Prupurtion of true edges
0.90%
0.65%
0.70%
1.15%
The evaluation framework described in Appendix B.2
operates search space of TF-TG pairs defined as Epotential .M x (M - 1), where M denotes the number of genes dataset. Within this space, edges present in the ground truth GRN are defined as &me. We quantified the distribution of Epotential and E Table 2 and found that there is an extreme class imbalance inherent to GRN inference across four datasets. For instance:
mESC:
Epotennall = 2,727,452, Eme = 24,557 (~ 0.9% positivity rate)
mHSC-L:
|Epoteniial 408,960,
Emell 4,705
(~ 1.1% positivity rate)
In such scenarios, AUROC disproportionately emphasizes the majority class (negative edges) due FP to its reliance on the false positive rate (FPR FP4TN) When |Emeel < |Epotemtial, the sum
FP TN ss Eppatentaal, which makes AUROC overly optimistic in evaluating model performance. Conversely, AUPRC is better equipped to handle such imbalanced scenarios [57]. As a result, AUPRC more accurately reflect the performance of the model.

---

C
Supplementary Contents of AUCell Algorithm
AUCell designed to quantify the activity of predefined gene regulatory regulons in scRNA- seq data. By calculating the Area Under the Recovery Curve (AUC) for regulons across in- dividual cells, it identifies cells exhibiting co- ordinated activation of specifie transcriptional programs, independent of absolute expression scales.
Top gene cutoff
S
AUC
For a regulon R comprising m genes and cell with detected genes, AUCell operates through three sequential phases. First, genes in cell are ranked by their expression values in descending order, generating an ordered list where Si denotes the
Gene rank
72
Figure 7: The recovery curve in the AUCell al- gorithm, with the gene ranking as x-axis and the recovery score the y-axis.
g" (gf ...... . - Ih),, highest-expressed gene. Ties in expression val- resolved stochastically avoid rank bias.
Subsequently, a binary recovery vector is constructed using an indicator function for regulon mem- bership:
TR( = [1, if of R 0.
(13)
otherwise
The cumulative recovery SRR, k) is computed by summing Tn( over the top k genes:
S(R, k)
In(If)
(14)
By taking the gene ranking i as the x-axis and the cumulative recovery score S(R, k) as the y-axis, a recovery curve can be plotted, as illustrated in Figure 7 Finally, the AUC score is obtained by calculating the area under this curve. In essence, the AUC score evaluates whether crucial subset of the input gene set is preferentially enriched among the top-ranked genes in each cell. It also quantifies the proportion of expressed signature genes and their relative expression levels compared to all other genes within the cell, thereby providing a measure the regulon's 's activity in that specific cell.
In KINDLE, after calculating the AUC score, additionally conducted analysis of variance this score. Based on statistical significance analysis, we ultimately identified 25 key TFs, with their corresponding p-values summarized in Table
Table 3: P-values and adjusted P-values for selected TFs, with those highlighted in red corresponding to TF's previously reported in the literature.
Gene
P-value
adjusted P-value
Gene
P-yalue
adjusted P-value
3.8203
10 10-18
4.4315
10-16
2.4402 x 10-50
3.9314
10-60
Kdmnb
8.1186
1.2302
67
Myb12
1.4571 x 10 48
1.8372
Sox2
1.2096 47
1.6705
10 47
Sux12
1.9448 x 10 19
2.3500
Nanog
2.5669 x 10
6.7672 10
Eisl
4.7752 x 10
9.2321 x 10
Utf1
2.4188 x 10
4.3836 x 10
Rest
6.14175 x 10
1,9808 x 10
Zíp42
7,8886 x 10 11-113
4.5754 x 10 10-112
Egrl
1.8206 x 10 10-82
5.2797 x 10-82
Earrb
2.3245 x 10
1.1235 10
Pml
6.8755 x 10
1.6616 10
Tpp53
3.8236 x 10
1.1088
Te7712
1.2169 x 10
1.7646
Sap30
3.0508 x 10-1101
4.4236
10 10-118
Trim28
1.8347 10 105
7.6010 X 10-
Nr0bl
1.2291 10-75
2.7419
10 -7
Elfi
1.9730 10-114
L-4304 10 -13
Níya
2.5722 -7
3.3907
10
Jarid2
5.9858 10 10-98
1,0211 10 -0-67
Tef12
119
8.6931
10 10-118
10-73
Akal
5.0038
25

---

D Supplementary Contents of In Silico Perturbation
In silico perturbation serves as a critical benchmark for evaluating the accuracy of GRN. By simulating TF perturbation (e.g., knockouts) and propagating their effects through the inferred GRN, this approach quantifies the network's 's ability to predict downstream gene expression changes and cell fate transitions. The following sections detail the computational framework of CellOracle's s in silico perturbation pipeline, which integrates GRN-based signal propagation, state transition modeling. and perturbation score calculation.
D.1 Signal Propagation for TF Perturbation Simulation
Given an inferred GRN represented by its regulatory coefficient matrix A MM where M denotes the number of genes. When perturbing a TF i, we set its expression to zero. By subtracting the perturbed expression values from the original ones, we obtain a vector Ax(() E M which is defined
A 00)
{
-i, Fi, 0, otherwise
if j = (TF knockout)
(15)
The impact of this perturbation on gene expression is propagated through the matrix A. For the first order perturbation, it is calculated
(x A A
(16)
where T based on the regulatory weights between gene pairs, propagates the perturbation effect to their direct targets. Higher order indirect effects are computed iteratively via K rounds signal propagation. Specifically:
Ax A Ax k = ... K -K=33 for defalut)
(17)
After the k-th propagation. the resulting perturbation vector (x is considered the simulated perturbation vector A It should be noted that during each propagation step, if any element in Ax(*) is less than 0, this element needs to be reassigned as 0, since gene expression levels are always non-negative. Mathematically, this can be expressed as:
(x max (x(()) (x), ())))
(18)
D.2 Cell-State Transition Estimation
The simulated gene expression shifts AXai are translated into cell-state transition probabilities through kernelized similarity analysis in the two-dimensional embedding space. For each cell i, the transition probability Pı, to its K-nearest neighbors (j E .V₂) is computed by comparing the simulated perturbation vector si with the observed expression difference X, X, This is formalized using a softmax function over Pearson correlation similarities:
exp (pl AX X, X,)/T)
Pi.j ΣREN, exp (pl ( X, X,)/T)
(19)
where o denotes the Pearson correlation function, X; means the expression of gene j. X; means the expression of gene i and 0.05 modulates the selectivity of the probability distribution. The transition probabilities are then projected onto the two-dimensional embedding space to construct a perturbation vector field. For each cell-neighbor pair, the coordinate difference vector Vi.j =V, -V; (V; means the coordinate of gene in the two-dimensional space) is weighted by Pi.j. yielding the simulated perturbation vector for cell i:
Vii
Σ Pi.j Vi.j
(20)
JEN,
This vector Vsim,i represents the predicted direction and magnitude of cell-state transition induced by the TF perturbation. To account for cellular heterogeneity, this process repeated across all cells, generating global perturbation vector field V E R where N is the number of cells. This vector field captures context-dependent regulatory effects, enabling systematic visualization of simulated differentiation trajectories.
26

---

D.3 Perturbation Score Calculation
The perturbation score (PS) quantifies the alignment between simulated perturbation-drinn driven cell- state transitions and intrinsic differentiation trajectories. The intrinsic differentiation vector field Vair NR2 is derived as the spatial gradient of pseudotime where pseudotime (inferred via diffusion pseudotime or RNA velocity) represents the progression of cells along developmental trajectories. Specifically, V aii, Vt, captures the direction and magnitude of natural differentiation for cell in the low-dimensional embedding space. To evaluate the impact of TFF perturbation, we compute the cosine similarity between the simulated perturbation vector sim, and the differentiation vector Vaiff.
Vsim, VdHIT,
PS;
(21)
V Vanr,
where a positive PS (green in Figure 4t.d) indicates that the perturbation promotes differentiation along the native trajectory, while negative PS (purple in Figure 4t.d) suggests suppression of differentiation. This directional alignment metric enables systematic identification of TFs that act as drivers brakes in cell fate determination.
27

---

E Supplementary Contents of Whole TF's AUC Score
Nanog
03
6.5
433
4.50
-030
-.00
Ztp42
Esrrb
Trp53
Sap30
Nr001
6150
-1.00
6.0
-600
-0.15
209
Niya
Bcl
AMI1
6.4
618
1.5
-2012
-05
8150
-4:5
Mybi2
Rest
4.05
-8.05
Pml
Tcf112
Trim28
EIM
Jarld2
-0
-12
Figure 8: Temporal dynamics of AUC scores for all 25 TFs identified by KINDLE. Blue scatter points represent AUC scores at individual time points, while the red curve denotes a quadratic fitting of these scores.
F Supplementary Contents of KINDLE Algorithm
28

---

Algorithm 2 KINDLE Framework for GRN Inference
1: Input:
2: Temporal expression matrix: G E RRNM with N time points and M genes
3: Spatial prior mask for teacher model: Mapatial E {0, 10 MMM
4: Output: Inferred GRN adjacency matrix gpred R. M
5: // Teacher Training Stage, Initialize Teacher for with parameters θr
6: for epoch [1, Emax] do
7: Sample batch B ~ G where [8] =B
8: for each sequence X (6) +: do
T: Historical window, W: Prediction window
9:
Partition ( (x(+)
T41T+H & Partition sequence into historical and future parts
Qt, Kr QKVQQV (X 1:T)
Get Q, K.V
A. softmax Q.K."
MMemporal
Compute temporal attention
vdg
H, A.V; E BBBIXM
Obatin input for spatial layer > t/: Transposition operation
13:
Q++ K,, V. (KV(((H.))))
14:
A, softmax
3 Mapootial
Vdh
( (AAVv))W, E BRBxWXM
>
W
Weights for output layer
BWM Σ 800 1000000
16:
Update Or Ver
Update teacher model parameters
17:
end for end for
18:
19: return Teacher model fr θr)
// Student Distillation Stage, Initialize Student fas with parameters θ and frozen Teacher for
21: for epoch E [1, Edsain do
for each (() 1.T+W B do
Partition (6) + (x 1:71 XT41T4W)
23:
(-11) (x(
24:
1.7)
Get teacher predictions
D(0 fs(X 1:T)
25:
>> Get student predictions
Spred BWM Σ 11
1
> Compute student prediction loss Choose distillation loss
27:
if Distillation Type "ardd" then
|1 (1))
28:
Lasstill BWA 37
29:
else if Distillation Type "Soft" then
p(0
30:
Raisill = BW 37 Σ
KL o
D 0: Softmax temperature
31:
else if Distillation Type "Bilinear" then (V) (P())
32:
dissill = BW M else if Distillation Type
33:
"Gaussian" then
19.9 8
34:
Laisill BWM Σ exp
2x
end
36:
Liotal = aCpree + (1 - Cdistil Update Og x Ve, Crotal
D a. E [0. 1]: Knowledge distillation coefficient
Update student model parameters
38:
end for
end for
40: return Student model fs(:; Os)
41: GRN Inference Stage
Partition G into H subsequences (S(h) Jh=1 with stride T
Initialize adjacency confidence matrix A MxM
44: for each ((h) (8(1)) M do
45:
À Ā 1 (
d: Extract attention matrix in student's spatial layer
46: end for
return Gpred = (A)
Select top edges
29


Reviews:
[{"review_id": "QjxAqxMUTe", "reviewer": "Reviewer_UYaC", "summary": "The paper presents a novel framework for learning the adjacency matrix of a gene regulatory network. Leveraging privileged-feature distillation, it transfers knowledge from a teacher model that has access to a prior network. Overall, the approach is both technically sound and conceptually interesting.", "questions": "See above", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 1, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nThe paper proposes an interesting idea of using privileged feature distillation, which is used a lot in the recommendation system, both incorporating prior network knowledge and not limited by them. According to the results, the framework shows good performances.\n\n**Weakness**\n\n1. Though leveraging the idea of privileged distillation information is interesting, why do we have to distill that? In terms of the recommendation system, privileged features are not seen during the inference, since we have to recommend items before knowing the click rate. However, I didn’t observe the necessity of using it in regulatory network inference. In my understanding, we can still observe the prior network during inference. Is there any offline/online setting? Or any other reasons? \n\n2. Following the above, if we look at Equation (4), it consists of both prediction loss and distillation loss. If the prior network is available during the inference, could the distillation loss be simply replaced by a regularization term that directly minimizes the distance between student learned A and prior M? It would be helpful if the authors could clarify whether they have any justification—either intuitive, theoretical, or empirical—for preferring distillation over direct regularization. Is there any additional information beyond the prior network M that is transferred from the teacher model during distillation?\n\n3. In line 189-190, the author mentions that they select top-k from the adjacent matrix, and k is exactly the same as the ground truth. Is there any risk of information leakage, since the users have to first know the number of correct connections? Do other baselines rely on this setting? If setting a threshold (e.g., 0.5), does that still work well?\n\n4. In terms of the writing, Equation (4) should be more clear. Inside the regulatory distillation loss, both $\\theta_S$ and $\\theta_T$ use same input, but teacher model should include an additional term representing prior network. In line 97, “hypnosis” should be “hypothesis”?", "was_revised": true, "final_justification": "The explanation by authors that incorporating prior may hurt the model's capacity to discover novel regulatory interactions is compelling and partly solve my problems.\n\nNow I understand and agree with the rationale, but further stronger quantitative evidence may be even better to show incorporating prior may hurt the performance. \n\nOverall, this rebuttal has addressed my main concerns. I will revise my score to borderline accept.", "rebuttal": "**We sincerely appreciate your thoughtful and valuable feedback. We will address the concerns and questions point by point in the following responses.**\n\n---\n\n### ***Question 1***\nThough leveraging the idea of privileged distillation information is interesting, why do we have to distill that? In terms of the recommendation system, privileged features are not seen during the inference, since we have to recommend items before knowing the click rate. However, I didn’t observe the necessity of using it in regulatory network inference. In my understanding, we can still observe the prior network during inference. Is there any offline/online setting? Or any other reasons?\n\n### ***Our response***\n### **1. Clarification on prior knowledge usage during inference**\nWe emphasize that our inference phase does not incorporate prior knowledge, consistent with the reviewer's observation that \"*privileged features are not seen during inference*\" in recommendation systems. As detailed in **Lines 54–56** of our manuscript, the student model is trained without prior knowledge. Further, **Lines 56–58** explicitly state that during inference, the student model **exclusively accepts gene expression data as input to infer a prior-free GRN**. Additionally, Figure 2 visually demonstrates that prior knowledge is solely provided to the teacher model, not the student model, and only gene expression matrices are input during inference.\n\n### **2. Rationale for knowledge distillation**\nThe motivation for distillation is elaborated in **Lines 30–43 and Figure 1**. Specifically, Inferring GRNs without priors involves an exponentially large search space (with $M$ genes, there are $M*(M−1)$ gene pairs to evaluate). While prior-based methods reduce this space to the $K$ edges provided in the prior knowledge to improve model performance, they introduce two critical issues: \n- The accuracy of the prior knowledge network will severely affect the model's performance, which depends on the overlap between the prior knowledge and the ground truth.\n- The inferred GRN is **strictly limited to a subset of the $K$ prior edges**, preventing discovery of real regulatory relationships outside this set. Given that priors are empirical and often contain high false-positive rates, this approach inherently caps model performance. \n\n**Therefore, we propose utilizing a distillation approach to address the limitations inherent in prior-based methods**. Our teacher model incorporates a prior knowledge network to learn comprehensive gene regulatory relationships. Subsequently, these learned features are input into the student model. **Crucially, the student model operates without requiring prior knowledge input. Instead, it learns the gene regulatory relationships by distilling knowledge from the features provided by the teacher model**, thereby enabling prior-free GRN inference. The self-attention computation within the student model operates without any masking mechanism (refer to the difference in the spatial layers between the teacher and student models in Figure 2). **This enables the student model to compute interactions between all possible gene pairs rather than be restricted in the $K$ edges from prior**. Since the student model does not receive prior knowledge input, it inherently avoids the two problems associated with prior networks. Furthermore, by employing the distillation strategy, we ensure the student model maintains robust inference accuracy.\n\n---\n\n### ***Question 2***\nFollowing the above, if we look at Equation (4), it consists of both prediction loss and distillation loss. If the prior network is available during the inference, could the distillation loss be simply replaced by a regularization term that directly minimizes the distance between student learned A and prior M? It would be helpful if the authors could clarify whether they have any justification—either intuitive, theoretical, or empirical—for preferring distillation over direct regularization. Is there any additional information beyond the prior network M that is transferred from the teacher model during distillation?\n\n### ***Our response***\n### **1. Clarification on prior knowledge usage during inference**\nAs elaborated in our response to Question 1, the prior knowledge network is unavailable during the inference phase. The rationale for employing the distillation loss is also detailed therein.\n\n### **2. Justification for preferring distillation over direct regularization**\nThe regularization term suggested by the reviewers, which directly minimizes the distance between the adjacency matrix $A$ learned by the student model and the prior matrix $M$, represents a valid approach for model optimization. However, **we contend that the regularization term is unable to address the fundamental limitations inherent in prior-based methods (Lines 38-43)**. Specifically, the teacher model incorporates the prior knowledge network as an input during training, it utilizes this prior as a mask within the Transformer's self-attention mechanism. This mask constrains the attention calculation exclusively to gene pairs specified within the prior network. Consequently, if the student model's learned $A$ is forced to approximate the prior $M$, **the resulting regulatory relationships would remain confined to the $K$ edges defined by the prior**. This confinement does not resolve the core limitations of prior-based methods. Precisely for this reason, we adopted the distillation strategy. The student model does not receive the prior as input, instead, it learns from the features distilled by the teacher model. This approach enables the beneficial propagation of prior knowledge to the student while simultaneously circumventing the constraints imposed by the prior's limitations.\n\n### **3. Temporal causality will be transferred to student model**\nBeyond the prior knowledge network $M$, the temporal causal relationships between genes learned by the teacher model are also distilled to the student model. The mechanism by which the teacher model learns these temporal causal associations through the temporal layer is detailed in **Lines 50-53 and 113-116**. Specifically, for time point $t$, a temporal mask is enforced to constrain each gene to attend only to information from preceding time steps ($< t$). Within the self-attention mechanism, attention weights between genes across these historical time steps are computed. Based on these weights, information from all time steps prior to $t$ undergoes weighted aggregation. This enables the representation of a gene at the current time step $t$ to integrate relevant historical context, thereby learning temporal dependencies among genes and ultimately leading to more accurate predictions of future gene expression.\n\n---\n\n### ***Question 3***\nIn line 189-190, the author mentions that they select top-k from the adjacent matrix, and k is exactly the same as the ground truth. Is there any risk of information leakage, since the users have to first know the number of correct connections? Do other baselines rely on this setting? If setting a threshold (e.g., 0.5), does that still work well?\n\n### ***Our response***\nWe emphasize that selecting the top $K$ edges **constitutes the standard procedure for obtaining the final inferred GRN within the GRN inference field and all baseline methods rely on this setting**. Crucially, this procedure does not introduce information leakage. The selection occurs solely during the final inference stage, **the value of $K$ itself is not incorporated into the model's training process**. Its function is analogous to setting the number of clusters in clustering algorithms, which determines the final output structure without influencing the core learning algorithm.\n\n---\n\n### ***Question 4***\nIn terms of the writing, Equation (4) should be more clear. Inside the regulatory distillation loss, both $\\theta_{S}$ and $\\theta_{T}$ use same input, but teacher model should include an additional term representing prior network. In line 97, “hypnosis” should be “hypothesis”?\n\n### ***Our response***\nWe sincerely thank the reviewer for identifying these two points. Specifically, we acknowledge that Equation (4) should explicitly reflect the teacher model's requirement for the prior knowledge network as an additional input, and we confirm that the term in Line 97 should correctly be \"hypothesis\". We apologize for these errors and confirm both will be rectified in the final manuscript.\n\n---\n\n**We hope the responses provided above adequately address the reviewer's concerns. If there are any aspects that remain unclear, please do not hesitate to contact us further, we remain fully available to provide additional details.**"}, {"review_id": "nuSBNcuGKm", "reviewer": "Reviewer_7DWh", "summary": "The authors proposed a deep learning framework, KINDLE, to infer gene regulatory network (GRN) using teacher -student modeling and knowledge distillation with gene expression information, temporal gene expression profiles and known gene-gene regulation. During test phase on new data, only gene expression profile is needed. The four variants of the proposed approach achieve the highest performance in terms of area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and F1 score on four data sets: Mouse embryonic stem cell (mESC), Mouse embryonic stem cell (mESC), Erythrocyte (mHSC-E), Granulocyte-Monocyte (mHSC-GM), and Lymphocyte (mHSC-L), compared with baseline methods that utilize or do not utilize prior knowledge.", "questions": "1. How does prior information help in the framework? Or what process is to be adapted to minimize the potential bias distillation process?\n2. In common biological experiments, there are some conditions with a few replicates for each condition. Please discuss the potential adoption of the current framework to this type of data.\n3. There are more improvement in terms of AUPRC, compared with AUROC. Can the authors explain what could be the reason?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The submission is technically sound.  The theoretical analysis and experimental results are well supporting the authors' claim. The methods used are appropriate. The work is almost complete piece of work. The authors mentioned both strengths and weaknesses of their work carefully and honestly.\n\nThe submission is clearly written and well organized. Some of the details may need more explanation. No source code was provided.\n\nThe authors use teacher - student model and knowledge distillation to improve gene regulatory network (GRN) inference. The framework achieves the highest performance comparing with some of the state of the art methods, given GRN inference is a difficult task. It is likely that when temporary gene expression data are available for the cell type of interest, the proposed approach will be used. \n\nThe work proves that teacher - student model and knowledge distillation improve GRN inference on the datasets tested, showing the importance of knowledge distillation. The authors clearly distinguish their contributions with previous contributions. The authors offer a novel combination of existing techniques, and the reasoning behind this combination is well-articulated.", "was_revised": true, "final_justification": "The authors answered question on what process is to be adapted to minimize the potential bias distillation process, on scenario with a few replicates for each condition, and on more improvement in AUPRC.", "rebuttal": "**We are deeply grateful to the reviewer for your recognition of our manuscript. We have carefully read your thoughtful and valuable feedback, and will provide point-by-point responses below to address your concerns and questions.**\n\n---\n\n### ***Question 1***\nHow does prior information help in the framework? Or what process is to be adapted to minimize the potential bias distillation process?\n\n### ***Our response***\n1. The prior information utilized in our framework is compiled from over 50 public data sources, consolidating regulatory pairs extracted from extensive experimental studies and literature. Within the transformer architecture, this prior functions as an attention mask during self-attention computation. It explicitly constrains the attention mechanism to focus solely on edges documented in the prior knowledge, while suppressing computation for irrelevant gene pairs. **Through this guided attention mechanism, the teacher model develops a global understanding of gene regulatory relationships**. The features learned under this prior guidance are subsequently distilled into the student model, enabling the latter to maintain high predictive performance without explicit prior constraints during inference.\n2. During experimentation, we observed that gene expression data exhibits inherent sparsity primarily attributable to technical limitations of sequencing protocols (e.g., dropout events where expressed genes yield zero counts). **This sparsity introduces systematic biases in raw expression values**. As both teacher and student models take these expression profiles as input during distillation, the technical biases become progressively amplified throughout the knowledge transfer process, ultimately compromising model performance. To mitigate this potential bias propagation, **we propose applying imputation algorithms as a preprocessing step**. These methods estimate biologically plausible expression values for technical zeros, thereby recovering expression levels obscured by sequencing artifacts, reducing systematic bias in input data, and ultimately enhancing the overall accuracy of the distillation process.\n\n---\n\n### ***Question 2***\nIn common biological experiments, there are some conditions with a few replicates for each condition. Please discuss the potential adoption of the current framework to this type of data.\n\n### ***Our response***\nThank you for raising this important point regarding the applicability of KINDLE to datasets with limited replicates, which is indeed a common scenario in biological experiments. We propose that KINDLE can adapt to such data through three potential solutions:\n### **1. Take advantage of consolidated prior knowledge**\nThe teacher model is trained on a consolidated prior network integrating experimentally validated regulatory relationships sourced from over 50 public databases and literature resources. Critically, these foundational datasets originate from large-scale experimental studies **inherently possessing substantial replicate samples**. When inferring GRNs for a target condition with sparse replicates, the scale and biological relevance of this integrated prior knowledge become paramount. By distilling patterns learned from these evidence-rich, **high-replicate sources**, the teacher model provides the student model with a robust foundation of general regulatory principles. This significantly reduces the student model's reliance on extensive replicates, effectively serving as a powerful biologically grounded regularizer to compensate for data sparsity in the target condition.\n\n### **2. Leverage appropriate data augmentation strategies**\n1. **Standard augmentation**: Applying standard data augmentation techniques to the limited expression profiles, such as adding biologically plausible noise.\n2. **Generative model-based augmentation**: Utilizing generative models (e.g., VAE, GAN and diffusion models) trained on relevant expression data to synthesize plausible additional expression profiles.\n\nIt should be noted that employing data augmentation requires careful validation to ensure biological fidelity and avoid introducing artifacts. Nevertheless, these strategies represent potential avenues within the KINDLE framework to further mitigate limitations imposed by scarce replicates.\n\n### **3. Transfer knowledge from biologically similar cell types**\nDuring teacher model training, data from biologically similar cell types exhibiting abundant replicate samples can be utilized. Knowledge distilled from these related, replicate-rich datasets is encoded within the teacher model and subsequently transferred to the student model. The student model then undergoes lightweight finetuning using the limited target-condition replicates. **This process mirrors the \"pretraining & finetuning\" paradigm**, enabling effective adaptation of the downstream model to specific data-scarce scenarios.\n\n---\n\n### ***Question 3***\nThere are more improvement in terms of AUPRC, compared with AUROC. Can the authors explain what could be the reason?\n\n### ***Our response***\nWe thank the reviewer for raising this point. We wish to emphasize that in **Lines 592-602**, we discuss why AUPRC is a more robust metric compared to AUROC. Specifically, let $\\mathcal{E}\\_{potential}$ represents the set of all explorable gene pairs and $\\mathcal{E}\\_{true}$ denotes the edges present in the ground truth network. As detailed in Table 2, which presents the distribution of positive and negative labels across the four datasets, we observe severe label imbalance in all datasets, with negative labels exceeding 99% of the total. In such scenarios characterized by extreme class imbalance, AUROC disproportionately emphasizes the majority class (negative edges). This occurs because AUROC relies heavily on the false positive rate ($FPR = \\frac{FP}{(FP + TN)}$). When $\\mathcal{E}\\_{true} \\leq \\mathcal{E}\\_{potential}$, the sum $FP + TN \\approx |\\mathcal{E}\\_{potential}|$, rendering AUROC overly optimistic in evaluating model performance. Conversely, as analyzed in the literature (i.e., *The Relationship Between Precision-Recall and ROC Curves*), AUPRC is better equipped to handle such imbalanced scenarios. Therefore, AUPRC serves as a more robust metric for assessing model performance under severe label imbalance than AUROC.\n\nConsequently, the significant improvements demonstrated by KINDLE in AUPRC (compared to the more modest gains in AUROC) provide stronger evidence for its superior performance over the baseline models.\n\n---\n\n**We hope the responses provided above adequately address the reviewer's concerns. If there are any aspects that remain unclear, please do not hesitate to contact us further, we remain fully available to provide additional details**."}, {"review_id": "uw6BlG3mmr", "reviewer": "Reviewer_2Z9D", "summary": "This paper introduces KINDLE, a framework designed to enhance the accuracy of GRN inference by eliminating fixed prior network information. The authors test this three-stage framework on both the mouse embryonic stem cell and the mouse hematopoietic stem cell development.", "questions": "Can you provide reference(s) for the claim made on page 1, lines 31 -33 regarding the quadratic scaling?", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "The paper clearly explains details and makes it easy to follow along.", "was_revised": false, "final_justification": "", "rebuttal": "We are deeply grateful to the reviewer for your recognition of our manuscript. We have carefully read your thoughtful and valuable feedback and will provide point-by-point responses to address your concerns and questions. \n\nRegarding the references for quadratic scaling in lines 31-33, **we primarily drew upon BEELINE [1] and a comprehensive review on GRN inference [2]**. BEELINE establishes a benchmark framework for GRN evaluation and proposes various metrics to assess the accuracy, robustness, and computational efficiency of GRN inference methods. The review article provides a systematic examination of the field, encompassing single-modal and multi-modal inference approaches, downstream GRN analyses, as well as current limitations and future research directions.\n\nBoth references offer authoritative summaries of the GRN inference landscape, delivering valuable insights into methodological approaches, persistent challenges, and evaluation strategies within the field. To strengthen reader comprehension, we will explicitly add citations for these two articles in lines 31-33.\n\n[1] Aditya Pratapa, Amogh P Jalihal, Jeffrey N Law, Aditya Bharadwaj, and TM Murali. Benchmarking algorithms for gene regulatory network inference from single-cell transcriptomic data. Nature methods, 17(2):147–154, 2020.\n\n[2] Pau Badia-i Mompel, Lorna Wessels, Sophia Müller-Dott, Rémi Trimbour, Ricardo O Ramirez Flores, Ricard Argelaguet, and Julio Saez-Rodriguez. Gene regulatory network inference in the era of single-cell multi-omics. Nature Reviews Genetics, 24(11):739–754, 2023.\n\n---\n\n**We hope the responses provided above adequately address the reviewer's concerns. If there are any aspects that remain unclear, please do not hesitate to contact us further, we remain fully available to provide additional details**."}, {"review_id": "DYnut6A5rh", "reviewer": "Reviewer_foG6", "summary": "The paper proposed a method called KINDLE for gene regulatory network (GRN) inference. KINDLE solves the limitation of relying on biologically plausible gene-gene interactions observed in prior knowledge. KINDLE distills encoded prior knowledge by a Teacher model into the Student model not to rely on any prior.", "questions": "Please see the Weaknesses part and the following additional questions:\n\n1. What happens if we set T=1 and W=1 in the equations and some of the experiments? How does this affect the output and efficiency of the model?\n2. In what manner does KINDLE manage to detect gene interactions that are not included in the prior knowledge dataset?\n3. Could you elaborate on the process of chunking the T time steps? What criteria or methodology are used for this chunking?\n4. How many GPU hours were required for training KINDLE? How does the computational cost of KINDLE compare to other baseline methods in terms of efficiency and resource consumption?\n5. A visualization of which regulatory insights the teacher transfers to the student would be helpful.", "limitations": "Please see weaknesses.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n- The student model achieves SOTA performance, demonstrating that causal regulatory knowledge is effectively compressed into a simpler architecture.\n- Temporal ensembling of attention matrices (Eq. 11) smooths stochastic noise in single-cell data, improving edge-calling reliability.\n\n**Weaknesses**:\n\n1. \"W\" is an important variable to evaluate the causality aspect of prediction, but was not explicitly specified in experiments. An ablation on W would really help to compare causal prediction ability to baselines.\n3. Gaussian RBF distillation outperforms other losses but requires careful tuning of λ (kernel width). No analysis shows robustness to λ variation.", "was_revised": true, "final_justification": "The authors have notably addressed the majority of my concerns, especially those related to new interactions. However, the extra analysis and ablation studies I anticipated were absent in their rebuttal, although the training time is reasonable to run additional experiments during the rebuttal week.\nI will keep my current rating score, 4 - Borderline Accept.", "rebuttal": "**We sincerely appreciate your thoughtful and valuable feedback. We will address the concerns and questions point by point in the following responses.**\n\n---\n\n### ***Question 1***\n\"W\" is an important variable to evaluate the causality aspect of prediction, but was not explicitly specified in experiments. An ablation on W would really help to compare causal prediction ability to baselines.\n\n### ***Our response***\nWe apologize for the oversight of not explicitly specifying the variable $W$ in the experiments. During the KINDLE training process, we conducted experiments with five distinct values of $W$ (1, 2, 4, 8, and 16). Among these, $W = 16$ yielded the optimal results, which are the results reported in the paper. We will add a clarification regarding the setting of $W$ in Lines 291-296 to enhance reader comprehension.\n\n---\n\n### ***Question 2***\nGaussian RBF distillation outperforms other losses but requires careful tuning of λ (kernel width). No analysis shows robustness to λ variation.\n\n### ***Our response***\nWe wish to emphasize that prior to applying the Gaussian RBF kernel, the input features underwent standardization ($\\mu = 0$, $\\sigma = 1$). **This preprocessing step substantially mitigates the model's sensitivity to $\\lambda$ by ensuring distances in the feature space are on comparable scales**. We will add clarification regarding the $\\lambda$ setting in Lines 291-296 to enhance reader comprehension.\n\n---\n\n### ***Question 3***\nWhat happens if we set $T=1$ and $W=1$ in the equations and some of the experiments? How does this affect the output and efficiency of the model?\n\n### ***Our response***\nSetting both $T$ and $W$ to 1 **enables the inference of a GRN for each individual cell**, thereby capturing single-cell resolution dynamic GRNs. Specifically, since each cell is assigned a pseudotime, each time step corresponds to a single cell. Under the configuration where $T = 1$ and $W = 1$, KINDLE takes the gene expression profile of the cell at the preceding pseudotime step as input and predicts the expression profile of the subsequent cell. Consequently, the model effectively learns the GRN associated with the preceding cell. **This approach allows us to infer the GRN for every cell along the developmental trajectory**. **By temporally aligning the GRNs inferred for each cell, we can observe the dynamic evolution of GRNs throughout development**. For instance, focusing on the regulatory interaction between Gene A and Gene B, their inferred regulation strength within each cell-specific GRN varies. This enables the reconstruction of the temporal trajectory of the regulatory strength for this gene pair, allowing observation of up- or down-regulation at specific developmental phases, thereby yielding deeper mechanistic insights into underlying molecular processes.\n\nAs noted in our response to Question 1, we evaluated five distinct values for $W$ (1, 2, 4, 8, and 16). However, we observed that the performance achieved with $T = 1$ and $W = 1$ was inferior to that obtained when both parameters were set to 16. We attribute this primarily to technical limitations inherent in single-cell RNA sequencing, which introduce significant noise and bias into the gene expression data. A key challenge is data sparsity, often manifested as dropout events (genes with actual expression that are erroneously recorded as zero counts due to insufficient sequencing depth). **When $T$ and $W$ are both set to 1, this noise and sparsity disproportionately impair model performance**. In contrast, increasing the number of input cells processed by the model (e.g., setting $T$ and $W$ to 16) mitigates these effects by effectively averaging out stochastic noise. This leads to enhanced model robustness and optimal performance. We will provide a more detailed explanation regarding the impact of $T$ and $W$ settings on model behavior in Lines 297-315 to improve reader comprehension.\n\n---\n\n### ***Question 4***\nIn what manner does KINDLE manage to detect gene interactions that are not included in the prior knowledge dataset?\n\n### ***Our response***\nWe precisely employ privileged-feature distillation to ensure KINDLE detects gene interactions beyond those included in the prior knowledge. Specifically, prior-based methods constrain the model to learn only within the $K$ edges provided by the prior knowledge, precluding the inference of gene interactions out of prior. To leverage the prior knowledge while overcoming this fundamental limitation, we adopt a distillation strategy. The teacher model incorporates the prior network as input, which acts as a mask within the Transformer's self-attention mechanism. This mask restricts attention computation exclusively to the $K$ edges specified by the prior knowledge. The knowledge learned by the teacher model from the prior is then distilled into the student model. Critically, the student model does not receive the prior knowledge as input. Instead, it learns features transferred from the teacher model. Furthermore, **the self-attention computation within the student model operates without any masking mechanism** (refer to the difference in the spatial layers between the teacher and student models in Figure 2). **This enables the student model to compute interactions between all possible gene pairs rather than be restricted in the $K$ edges from prior**. Thus, while learning the features distilled from the teacher, the student model simultaneously retains the capacity to detect gene interactions absent from the prior knowledge, thereby achieving high inference accuracy and resolving the inherent limitation of prior-based methods.\n\n---\n\n### ***Question 5***\nCould you elaborate on the process of chunking the T time steps? What criteria or methodology are used for this chunking?\n\n### ***Our response***\nWe apologize for the omission of a detailed description regarding the process of chunking the $T$ time steps. We will now provide a comprehensive explanation of this procedure. let $G \\in \\mathbb{R}^{N \\times M}$ denotes the temporal single-cell expression matrix, where $N$ represents temporally ordered cellular states and $M$ is the number of genes. our time series data $G = [\\mathbf{e}_1,\\ldots,\\mathbf{e}_N]^T$ possesses the temporal relationships of genes during the differentiation process as we map each cell to a pseudotime (**each cell corresponds to a single time step**), $\\mathbf{e}_i \\in \\mathbb{R}^{M}$ represents the gene expression at the $i$-th time step. After that, we split $G$ into dataset $\\mathcal{D} = ((\\mathbf{X}_i,\\mathbf{Y}_i)|i = 1,\\ldots,L)$, where $\\mathbf{X}_i$ are input samples, $\\mathbf{Y}_i$ are the ground truth associated to each sample and $L$ is the number of samples. This yields the assignments that $\\mathbf{X}_i \\coloneqq G[i:i+T,:]$ and $\\mathbf{Y}_i \\coloneqq G[i+T:i+T+W,:]$, where $G[a:b,:]$ denotes the row submatrix from index $a$ to $b-1$, capturing $T$ and $W$ consecutive cells respectively. **In conclusion, we construct input $\\mathbf{X}$ and ground truth $\\mathbf{Y}$ through sliding-window sampling on the temporally ordered cell matrix $G$, utilizing adjacent windows of length $T$ and $W$ respectively**.\n\n---\n\n### ***Question 6***\nHow many GPU hours were required for training KINDLE? How does the computational cost of KINDLE compare to other baseline methods in terms of efficiency and resource consumption?\n\n### ***Our response***\nWhen $T$ and $W$ are set to 16, KINDLE requires an average of 20 minutes to complete 30 training epochs across all four datasets on an 80GB NVIDIA A100 GPU, with total GPU memory consumption of 30GB. **KINDLE is more efficient compared to prior-based baselines CEFCON and Celloracle, while maintaining comparable runtime to NetREX**. Specifically, CEFCON employs a graph neural network architecture that incurs higher computational complexity than KINDLE's Transformer framework and the requirement to compute both a real network and a false network for contrastive learning substantially increases its training time (approximately 1 hour). Celloracle's execution time is often prolonged due to the computationally intensive process of constructing prior networks from scATAC-seq data, which can take several hours to days. Additionally, its GRN inference relies on iterating through each gene to perform ridge regression, resulting in low efficiency.\n\n**Regarding memory usage, KINDLE exhibits comparable but marginally higher GPU memory consumption than the other methods**. This is attributable to the simultaneous loading and computation of both teacher and student models during training. We acknowledge this limitation entails a risk of memory overflow when processing large-scale datasets. To mitigate this, we plan to explore memory-optimized attention implementations (e.g., Flash Attention) in future optimization of KINDLE.\n\n---\n\n### ***Question 7***\nA visualization of which regulatory insights the teacher transfers to the student would be helpful.\n\n### ***Our response***\nWe agree that visualizing the features transferred from the teacher to the student model would enhance reader comprehension. However, under the current official regulations, we are not permitted to upload experimental results such as images or videos. We commit to making these visualizations available once permission is granted.\n\n---\n\n**We hope the responses provided above adequately address the reviewer's concerns. If there are any aspects that remain unclear, please do not hesitate to contact us further, we remain fully available to provide additional details.**"}]

Author rebuttal:

Meta-review: **Summary**  
This paper introduces KINDLE, a teacher–student knowledge distillation framework for gene regulatory network (GRN) inference. The teacher model integrates prior biological knowledge with temporal expression data, while the student model learns through distillation and performs inference without priors. This design aims to capture the benefits of prior knowledge while retaining flexibility to discover novel interactions. The method is evaluated on benchmark datasets and shows performance gains over baselines, with case studies suggesting biological relevance.

**Strengths**  
- Novel use of knowledge distillation to transfer information from a prior-informed model to a prior-free student model.  
- Empirical results are generally strong, with state-of-the-art performance across several datasets.  
- Biological case studies provide evidence that the method can yield meaningful insights.  
- Authors engaged constructively in rebuttal and discussion, clarifying several technical points.

**Weaknesses**  
- Some experimental concerns remain only partially addressed, including the need for additional ablations and clearer interpretability demonstrations.  
- Biological validation, while promising, could be expanded for stronger support.  

**Key reasons for decision**  
The reviewers broadly agreed that the paper is technically sound and makes a novel contribution, with convincing empirical results. Remaining concerns about missing experiments and interpretability temper the enthusiasm but do not undermine the main contribution. Given this balance, a poster-level acceptance is warranted.

**Discussion and changes**  
The rebuttal and discussion resolved some key questions. Reviewer UYaC’s concerns about excluding priors at inference were addressed by the authors’ explanation that priors limit discovery, which the reviewer accepted. Reviewer foG6’s questions on interpretability and feature transfer received detailed responses, though more empirical evidence would have been welcome. Reviewer 7DWh’s concerns on bias and evaluation were answered, but the review itself was not highly technical. Reviewer 2Z9D did not engage beyond an initial very brief Strong Accept despite multiple prompts, limiting the weight of that rating. Overall, the discussion improved confidence in the approach, but the uneven depth of reviewer engagement tempers the strength of the consensus.

**Decision: Accept (poster)**  
The paper presents a novel and relevant framework with strong empirical results, though interpretability limitations and uneven review depth reduce confidence in a higher recommendation.
Decision: Accept (poster)