dataesr/scientific-paragraphs-categorization huggingface dataset:
This constrain is particularly important, since removing it would lead to a degenerate behaviour, where each regulator would be regulated only by itself, which would prevent the system to infer interactions between regulators. In addition to the aforementioned constrains, further constrains could also be included in order to integrate information from external datasets, such as ATACseq, CHIPseq or Transcription Factor Binding Sites motifs. In practice, we address this task by training, for each gene g ∈ Γ, a linear regressor with meta-parameters θ and coefficients C ,g , to predict X c,g from the regulators' expressions X c,Ψ , for any condition c. In order to consider the models generalization capabilities, we relied on a nested cross-validation procedure  [41] , in order to avoid overly-optimistic estimations of the generalization capabilities  [42].
An inner cross-validation step is used to select suitable parameters for each regressor, that lead to higher generalization capabilities. Besides that, an outer cross-validation step is executed on conditions that were not used to train the model and tune the parameters. This second step allows to monitor the generalization capabilities of each regressor, and eventually set to zero the coefficients of regressors suffering from poor generalization capabilities. More formally, the gene expression matrix X is first divided in k outer non-intersecting equal-sized subsets X f old 1 ,Γ , X f old 2 ,Γ ,. X f old k outer ,Γ , such that ∀i ∈ {1,. , k outer } X f old i ,Γ describes the expression levels of all genes Γ in D/k outer conditions. Then, the outer crossvalidation defines iterativelly each subset X f old i ,Γ as test set to assess the generalization capabilities of the model, while the remaining k outer -1 subsets are concatenated and used as a learning dataset X learn,Γ for model selection and training.
The inner cross-validation subdivides X learn,Γ in k inner non-intersecting equal-sized subsets X f old l 1 ,Γ , X f old l 2 ,Γ ,. X f old l k inner ,Γ. Similarly, the inner cross-validation takes each subset X f old l j ,Γ ∀j ∈ {1,. , k inner } as validation set to select the best model, while the remaining subsets are concatenated and used to train models with different parameter settings, and the model with the parameters that exhibited the best average performance on the validation datasets is selected, and retrained on the entire learning dataset. In this work, the number of folds for the inner and the outer cross-validations where set to k inner = 5 and k outer = 5 respectively. In this work, we propose two Generalizable Gene Self-Expressive Network inference methods, called respectively GXN•OMP and GXN•EN.
GXN•EN Algorithm The second method proposed in this paper relies in the well-known ElasticNet regression technique  [36] , that addresses the linear regression task using simultaneously 1 and 2 regularization. More formally, GXN•EN addresses the objective function stated in Equation (  4 ). C * ,g = argmin C ,g X ,g -X • C ,g 2 2 2D + αρ C ,g 1 + α(1 -ρ) 2 C ,g 2 2 with C g,g = 0 ∀ g ∈ {1,. , N} and C j,g = 0 ∀ j / ∈ Ψ (4) According to  [36] , pure 2 regularization (Ridge regression) shrinks the coefficients of correlated features while keeping them non-zero, while pure 1 regularization (Lasso regression) tends to select only one feature among a set of correlated dimensions, and exhibits a degenerate behaviour on highly correlated datasets. ElasticNet is a compromise between Ridge and Lasso regression, and the trade-off between 1 and 2 penalties is governed by a mixing parameter ρ ∈ [0, 1], when ρ = 0 only an 2 regularization is used, and the task corresponds to a Ridge Regression, while for ρ = 1 only a 1 regularization is used, and the problem is equivalent to a Lasso Regression.
Relationship with GRN Inference Even if the self-expressiveness and the subspace preserving properties have not been explicitly mentioned in the context of GRN reverse-engineering, GRN inference methods based on feature selection on regression algorithms rely on related principles. Indeed, the central assumption of this paradigm is related to the self-expressiveness property, since it states that the expression X c,g of each gene g for any condition c, can be expressed as a function of the levels of expression X c,Ψ of regulator genes Ψ, for the same condition c. Regressors are trained by finding a set of parameters that minimize some kind of average regression error, on the training dataset. In practice, existing methods rely mainly on linear regressors (e. g. ,  [10] ) or regression trees (e.
g. ,  [9, 43] ) as base-learners. And they tend to train ensembles of regressors on randomized versions of the dataset (e. g. ,  [10] ), or on subsets/subspaces of the dataset (e. g. ,  [9, 43] ). Then, for each gene g, a feature importance vector γ (g) ∈ R |Ψ| is computed, such that γ (g) ψ denotes the average importance of regulator ψ ∈ Ψ in the prediction of X ,g. This measure is assumed to quantify the dependency between genes g and ψ. Notice that in this case, regressors are only used to score the links, and they do not model the genes expression quantitatively, as in the model-based family or in the formalism presented in this article. An important difference between the GXN formalism and existing regression-based GRN inference tools, is that the GXN formalism focuses on the predictive power of the generated models.
Datasets Description 2. 6. 1. DREAM5 Dataset In order to assess GXN•OMP and GXN•EN, and compare them to state-of-the-art approaches, we used the DREAM5 benchmark data  [24]. This benchmark has three datasets from real organisms, namely E. coli, S. aureus and S. cerevisiae, and an in silico simulated dataset. Each dataset contains a gene expression matrix, a list of TFs, and a list of known regulatory links between TFs and their TGs, i. e. , a gold standard GRN. The E. coli, S. aureus and S. cerevisiae gene expression matrices are Microarray datasets, downloaded from Gene Expression Omnibus platform (GEO) (http://www. ncbi. nlm. nih. gov/geo, accessed on 6 February 2023). The authors of  [24] , applied to these datasets a Robust Multichip Averaging background adjustment, a quantile normalization, a probeset median polishing and a logarithmic transformation.
Moreover, the lists of TFs for E. coli, S. aureus and S. cerevisiae, were determined by means of a Gene Ontology (GO) annotation conducted in  [24] , completed with manually curated TFs list from the RegulonDB 6. 8 database  [44]  for E. coli, and a list of TFs provided in  [45]  for S. cerevisiae. The gold standard E. coli GRN, includes only regulatory links with strong experimental evidence, from the RegulonDB 6. 8 database  [44]. The gold standard S. cerevisiae GRN, includes regulatory interactions determined in  [46] , through the study of ChIP-chip datasets and a TF binding sites motifs analysis. Regarding S. aureus, no experimentally validated GRN was available, nevertheless the set of prokaryotic regulatory interactions reported in the RegPrecise database  [47] , was used as a proxy of a GRN.
Therefore, for this organism the quality of the GRN is lower than those of the other DREAM5 benchmark datasets, and it was included in  [24]  as a complementary way to asses the methods' robustness. The in silico dataset, was generated using the GeneNetWeaver software  [48]. According to  [24] , the in silico GRN is a randomized version of the E. coli GRN, incorporating 10% of new random regulatory links. This GRN, was used to generate a gene expression matrix, using a dynamical system of Ordinary Differential Equations: GeneNetWeaver aims at modeling the dynamics of the concentration of mRNAs and proteins stored as two distinct numerical vectors. The concentration of each protein is determined linearly by the concentration of its corresponding mRNA and its translation rate. The concentration of regulatory proteins determine the rate of transcription of target genes into mRNA, through a gene network additive and multiplicative interaction model based on the thermodynamic assumption that each promoter of a target gene is occupied with a probability depending on the concentration of its corresponding regulatory protein and its dissociation constant, via a Hill equation.
Both mRNAs and proteins are subject to linear degradation. Finally, the GeneNetWeaver software simulates perturbations in the network via i) gaussian perturbation, i) knockdown and ii) knock-out in silico experiments, and relies on the Runge-Kutta 4-5 solver to simulate the evolution of the GRN dynamical system. We refer the readers to  [49]  for a detailed description of the GeneNetWeaver software and its underlying model.
RNA-seq Multi-Tissue Eukaryote Datasets Given the lack of gold-standard datasets from complex eukaryotic organisms for GRN inference assessment, we have collected three RNA-seq datasets from eukaryotic organisms, namely C. familiaris, R. norvegicus, and H. sapiens, to evaluate GXN•OMP and GXN•EN methods in more challenging scenarios. Each dataset contains a RNA-seq gene expression matrix, a Gene Ontologies annotations database and a list of regulators (including TFs and cofactors). The Gene Ontology annotations for each dataset were downloaded from the Ensembl database  [50] , and the lists of regulators were downloaded from the AnimalTFDB database  [51]. Regarding gene expression datasets, we downloaded the iDog C. familiaris database  [52] , this dataset contains a total of 75 conditions from 6 cell-types, namely 25 samples from MDCK cells, 4 from adrenal cortex tissue, 12 from heart tissue, 21 lymphoma cells, 9 from neuroretina cells and 4 from pituitary tissue.
In addition, we downloaded the rat BodyMap database  [53] , which contains 80 conditions from 11 tissues, namely 8 samples from the adrenal gland, 8 from the brain, 8 from the heart, 8 from the kidney, 8 from the liver, 8 from the lung, 8 from the spleen, 8 from the thymus, 8 from the gastrocnemius, 4 from the uterus, and 4 from the testis. Finally, regarding the H. sapiens dataset, we downloaded and concatenated three datasets from different cell-types, namely the Macrophage Immune response  [54]  which contains 89 samples, the iPSC-derived Sensory Neurons  [55]  which contains 106 samples, and the 465 lymphoblastoid cell-lines  [56]  which contains 462 samples. The four last datasets were downloaded from the Gene Expression repository  [57]. For these three organisms, and particularly for C.
familiaris and R. norvegicus, the number of samples per tissue is not sufficient to run tissue-specific network inferences. Therefore, in this work we decided to combine samples from different tissues, running a single inference per organism, in order to have a larger sample with more variability. Unlike in the DREAM5 dataset, gold standard GRNs were not available for the RNA-seq eukaryotic datasets. In this work, we decided to focus on the interactions between regulator genes only, and thus any gene that is not a TF or a cofactor, was not included in the analysis. More formally, let Ψ be the set of genes encoding TFs or cofactors, and let Γ be the total set of genes considered in the analysis, then for these datasets we set Γ = Ψ. The aforementioned datasets were carefully analyzed as described in Appendix A, and no important batch effects could be detected, as depicted in Figure  A1a-d.
RNA-seq Disease/Control Case Study In order to illustrate how the GXN methodology can be used to perform differential regulatory communities analysis, we applied the GXN methodology on a control/disease RNA-seq case study. In practice, we analyzed the publicly available "Allen Brain Institute Aging, Dementia and TBI study" dataset  [59]. This dataset contains batch-corrected RNAseq gene expression matrices, reporting the level of expression of H. sapiens genes in 377 samples from four brain regions (i. e. , temporal neocortex, white matter of forebrain, hippocampus and parietal neocortex), of 107 male and female elderly individuals from the Change in Thought (ACT) cohort  [60]. According to the Diagnostic and Statistical Manual of Mental Disorders (DSM) clinical diagnosis, available for each patient, most samples belong to two categories, namely No Dementia (i.
e. , control) donors (193 samples) and Alzheimer's disease type patients (110 samples). In order to focus on the most promising genes, we have filtered out genes that are not expressed in all samples. As described in the previous paragraph, we used the Ensembl database  [50]  H. sapiens The Gene Ontology annotations, as well as the AnimalTFDB database  [51]  lists of regulators for H. sapiens.
Evaluation Against DREAM5 Gold Standard The evaluation of the methods proposed in this article, against DREAM5 gold standards, was conducted as a binary classification task, where possible regulatory links are classified as true or false interactions, as in  [24]. Regulatory links between TFs and TGs that were found experimentally, and reported in the benchmark datasets were considered as true interactions (regardless whether they are Up or Down regulatory links), while pairs of TFs and TGs for which the experimental study could not reveal a regulatory interaction were considered as false interactions. Then, depending on the GRN inference method outcome, four possible scenarios exist for each link: (i) True Positive: the method infers a true link, (ii) False Negative: the method fails inferring a true link, (iii) False Positive: the method infers a false link as being positive, and (iv) True Negative: the method does not predict a false link.
Inner Evaluation Procedure As described in the previous section, when experimentally validated regulatory links are available, evaluation measures for binary classification (e. g. , AUROC, AUPR) should be used to evaluate GRN inference tools, then, we used this procedure for the DREAM5 datasets. Nevertheless, in order to assess inference methods on datasets lacking of gold stantard GRNs (i. e. , C. familiaris, R. norvegicus, H. sapiens datasets), we have decided to consider the following inner evaluation metrics.
The in-degree of g is defined as the number of regulators involved in the regression model predicting the expression of gene g, i. e. , deg -(g) = |{ψ ∈ Ψ | (ψ, g) ∈ E}|. Finally, let deg + (ψ) be the out-degree of the node representing a regulator gene ψ. This measure is defined as the number of target genes including ψ in their gene expression regression models, i. e. , deg + (ψ) = |{g ∈ Γ | (ψ, g) ∈ E}|. 2. 9. RNA-seq Eukaryote Datasets Community Analysis 2. 9. 1. Community Detection In order to deepen the analysis of the networks inferred on the RNA-seq Eukaryote dataset, and given the lack of gold standard GRNs available, we decided to assess whether the gene networks inferred tend to structure in communities (i. e. , sub-networks of genes densely intra-connected by regulatory links) sharing common functional roles.
GSEA and GO Enrichment Analysis In this work we used a Gene Set Enrichment Analysis (GSEA)  [67]  in order to determine communities of genes that are collectively over-expressed or under-expressed, with statistical significance, in a particular tissue or cell-type. In practice, we used the differenceof-classes ranking method, we assessed the result's significance running 10,000 gene-set permutations and only kept relationships with a p-value and a False-Discovery-Rate (FDR) both lower than 0. 05. Finally we extracted statistically over-represented GO terms for each community of genes, using the GOATOOLS  [68]  Python library. In practice, we used a default significance cut-off p-value equal to 0. 05, with a Benjamini Hochberg multiple test False Discovery Rate (FDR) correction, and we only kept the relationships with a FDR lower than 0.
Parameter Setting and Implementation In order to analyze, on the DREAM5 datase, the impact of a rather large range of parameter values, but focusing with more details on sparser solutions, we have run ex-periments with δ = {0. 01, 0. 03, 0. 05, 0. 1, 0. 2, 0. 3, 0. 5} for the OMP algorithm, and ∈ {0. 02, 0. 025, 0. 033, 0. 05, 0. 1, 0. 2} for the ElasticNet algorithm. Moreover, we illustrated the performances of GXN•OMP and GXN•EN, on the Eukaryotes RNA-seq datasets, with d max 0 = 30 and = 1/3 respectively. Both parameter settings tend to produce simple models, with few coefficients (and also rather lower R 2 scores). Therefore using such a parameter setting, allows to assess the GXN•OMP and GXN•EN models, on a worse case scenario, where sparser and simpler models are used to deal with complex eukaryotic organisms' RNA-seq datasets.
In order to study the communities within the GXN models we only retain the links from models exhibiting an R 2 determination score higher than 0. 5 for the validation set (i. e. , models that exhibit a higher enough generalization performance). The methods proposed in this paper were implemented using the GReNaDIne [69] package, and the Scikit-Learn  [70]  ElasticNetCV and OrthogonalMatchinPuirsuitCV implentations of the ElasticNet and OMP algorithms. Moreover, the evaluations were run using the GReNaDIne  [69]  package. The software to run the inferences and the analysis presented in this article is available at https://gitlab. com/bf2i/gxn (last updated on 20 February 2023). Important characteristics of DREAM5 and RNA-seq datasets studied in this article are summarized in Table  1. All experiments were executed on a 2,9GHz Intel Core i9 CPU, running macOS Big Sur 11.
2. 3, with a 32 Go RAM capacity. The GXN•OMP and GXN•EN models, exhibit in general high sparsity levels (> 65%), which depend both on the parameter settings and the dataset characteristics, as shown in Figure  2. Regarding the parameter settings impact, increasing δ in the GXN•OMP method and decreasing in the GXN•EN algorithm, allows to build more complex models, with lower sparsity (Figure  2a, d ), accordingly genes exhibit higher in-degrees (Figure  2c ,f), and regulators present higher out-degrees (Figure  2b, e ). Regarding the impact of the dataset characteristics, in general real datasets, and specially S. cerevisiae and E. coli datasets, tend to lead to more complex models (lower sparsity), with genes exhibiting larger in-degrees and regulators with higher out-degrees, while models inferred for the in silico dataset are sparser.
The S. aureus dataset exhibit also rather simple models, but since this dataset contains few regulators, the sparsity tends to be lower than those of more complex S. cerevisiae and E. coli models. In order to compare the sparsity obtained by GXN based methods, to those obtained by state-of-the-art techniques, we have extended the definition stated in Section 2. 8 to methods based on feature importance, by computing the sparsity as the percentage of feature importance scores that are close to zero. In practice, we consider scores lower than a threshold equal to 10 -5. As reported in Table  2 , state-of-the-art methods tend to exhibit low sparsity levels (< 1%), unlike GXN based methods, as described in Section 2. 5.
Regressors Performance In Figure  3 , we represent, for the different algorithms, datasets, and different parameter settings, the average R 2 determination coefficient, and training run-time, measured throughout the outer cross-validation procedure for each gene expression regression task. In general, for both GXN•EN and GXN•OMP, when the number of coefficients increases, the R 2 coefficient increases at first, and then tends to reach a plateau. Indeed, GXN•EN models exhibit significantly better results when decreases from 0. 2 to 0. 05, and tends to plateau for ≤ 0. 033. Similarly, GXN•OMP models exhibit higher R 2 scores when δ increases from 0. 01 to 0. 1, and reaches a plateau for δ ≥ 0. 1. Moreover, the R 2 scores obtained for each method with the same parameter setting, depend to an important extent on the analyzed dataset.
In practice, real world datasets lead to higher determination coefficients, while the results obtained for the in silico dataset tend to be poorer. Compared to the state-of-the-art results, the best GXN•EN models exhibit similar results to the Support Vector Machine Regressor (SVR) ones for the real datasets, and better results than those obtained by the Random Forest Regressor (RF) ones. While the best GXN•OMP models, despite being slightly worse than the SVR results, tend to exhibit comparable results with respect to the RF results, on the real datasets. Regarding the in silico dataset, both GXN•EN and GXN•OMP models exhibit lower determination coefficients than SVR and RF. Despite the high sparsity and simplicity of GXN•EN and GXN•OMP models, they exhibit mostly positive determination coefficients, with levels that are comparable with respect to more complex state-of-the-art tools.
Run-times required to fit each regressor, represented in Figure  3 , show that GXN•EN and GXN•OMP are rather fast methods, such as SVR, since they require in most cases between 10 -3 and 1 second to fit each TG regressor, depending on the parameter setting and the dataset. Whereas the RF is a very time consuming method, that requires between 10 -1 and 10 seconds to fit each TG regressor. Moreover GXN•OMP algorithm is faster than GXN•EN, since in most cases GXN•OMP fits each TG regressor in less than 10 -1 seconds, while GXN•EN requires between 10 -1 and 1 second in most cases.
GRN External Evaluation Figure  4  represents, for the different algorithms, and different parameter settings, the AUROC and AUPR evaluation scores on the DREAM5 datasets. In general, for both GXN•EN and GXN•OMP, increasing the models complexity tends to lead to slightly better scores, and then tends to reach a plateau. Indeed, the GXN•OMP models tend to exhibit slightly better AUROC and AUPR scores when δ increases, nevertheless this trend is small, and scores are mostly stable. Similarly, for the real dataset, GXN•EN GRNs exhibit higher, or at least equal AUROC and AUPR scores when decreases from 0. 2 to 0. 033 and then the trend tends to plateau for ≤ 0. 1, but the method exhibits an opposite trend, for the in silico dataset. Moreover, the best results obtained by the GXN•OMP and the GXN•EN methods are comparable with respect to those obtained by SVR and RF.
Indeed, the average AUROC score obtained, on the DREAM5 datasets, by GXN•EN (with = 0. 02), SVR and GXN•OMP (with δ = 0. 5) are very similar (respectively 0. 725, 0. 71 and 0. 7), while RF exhibited a lower average AUROC = 0. 67. In terms of average AUPR, both GXN•EN and GXN•OMP obtained the highest average score AUPR = 0. 26, outperforming SVR and RF, which obtained respectively 0. 235 and 0. 22.
RNA-seq Multi-Tissue Eukaryotic Datasets The networks inferred from the C. familiaris, R. norvegicus and H. sapiens multi-tissue RNA-seq datasets exhibit a high sparsity level (≥ 99%), as reported in Table  3. The median number of regulators involved in the control of each gene, determined the GXN•EN method, is close to 10 regulators for all datasets, as shown in Figure  5b. While, using the same parameter setting, the GXN•OMP determined a similar number of regulators for the C. familiaris and R. norvegicus datasets, but has built, for the H. sapiens dataset, models with twice more regulators in median, as shown in Figure  5b. This suggests that the GXN•OMP is more flexible and better adapts to the particularities of each dataset. Moreover, these networks exhibit a modular structure, they comprise communities of genes that share more connections between them, than with genes from other communities.