dataesr/scientific-paragraphs-categorization huggingface dataset:
However, this is only possible if 1) the data model is well described, as part of the metadata associated with the dataset, 2) if the metadata follows standards as much as possible, and 3) is supported by the creation of meta-services, which index all relevant data sources so that the customized data services can find them. For the latter, Google Dataset Search could, in principle, be used but we believe that domain-specific solutions would be more beneficial since they can profit from community-endorsed standards like specialized unique identifiers (NInChI, European Registry of Materials) and preselection of the most relevant sources. 4. What we describe in the first 3 points requires an ecosystem with many components including, but not limited to, repositories of documented data standards/models, templates and checklists to guide (meta)data collections, data input tools (like electronic lab notebooks, workflow visualization (instance maps), and data transfer standards), data storage solutions, data indexing solutions listing relevant data resources, and customized data retrieval solutions for the different data users as well as general and domain-specific unique identifiers, ontologies and other FAIR enabling and supporting resources.
Acknowledgments The authors acknowledge  Barry Hardy  ( Edelweiss Connect ) and  Eugenia Valsami-Jones  ( University of Birmingham ) from the  ACENano  project for useful discussions, as well as the following NanoCommons partners and Transnational Access users for discussions that supported the refinement of the concepts presented herein:  Andrea Haase  ( BfR ),  Claus Svendsen ,  Lee Walker  and  Amaia Green Etxabe  ( UK CEH ),  Georgios Konstantopoulos ,  Nikos Nikoloudakis  ( NTUA ),  Konstantinos Kotsis  ( UCD ),  Martin Himly  ( PLUS ),  Beatriz Alfaro  and  Andreas Falk  ( BNN ),  Jan Schüürman  and  Frank von der Kammer  ( Uni Vienna ).
Grouping gene expression into gene set activity scores (GSAS) provides better biological insights than studying individual genes. However, existing gene set projection methods cannot return representative, robust, and interpretable GSAS. We developed NetActivity, a framework based on a sparsely-connected autoencoder and a three-tier training that yields robust and interpretable GSAS. NetActivity was trained with 1,518 well-known gene sets and all GTEx samples, returning GSAS representative of the original transcriptome and assigning higher importance to more biologically relevant genes. Moreover, NetActivity returns GSAS with a more consistent definition than GSVA and hipathia, state-of-the-art gene set projection methods. Finally, NetActivity enables combining bulk RNA-seq and microarray datasets in a meta-analysis of prostate cancer progression, highlighting gene sets related to cell division.
When applied to metastatic prostate cancer, gene sets associated with cancer progression were also altered due to drug resistance, while a classical enrichment analysis identified gene sets irrelevant to the phenotype. incorporating topological information from gene-gene co-expression networks  11, 12  , or modeling the propagation of signals within pathways 13. As the GSAS' representativity is maximized within each dataset, it may lead to potential differences in the most relevant genes across datasets due to overfitting, compromising the GSAS robustness. Further, they present challenges in terms of interpretation, as researchers cannot readily determine which specific genes are more relevant for GSAS computation. Shallow sparsely-connected autoencoders pose as ideal alternatives to address the shortcomings of current gene set projection methods  [14] [15] [16].
We trained the model by selecting gene sets from Gene Ontology (GO)  17, 18  Biological Processes and KEGG  [19] [20] [21]  pathways and using the entire GTEx project 22 data, showing that NetActivity generates GSAS independent of the initialization parameters that translate to unseen datasets representing different conditions. Further, NetActivity returns an importance score for each gene that agrees with their biological relevance in the studied context. The main model is distributed in a Bioconductor package to facilitate the computation of GSAS on new datasets. We compared NetActivity with GSVA and hipathia, two widely used state-of-the-art gene set projection methods, and found that NetActivity had the highest overall performance. Finally, we demonstrated some applications of NetActivity by (1) performing a meta-analysis in prostate cancer combining datasets from three different gene expression platforms; and (2) finding new biological insights in a metastatic castration-resistant prostate cancer dataset.
Introduction Although gene expression analyses have greatly improved our understanding of the physiopathology of multiple diseases and conditions, gene expression analyses performed at the gene level can be difficult to interpret, particularly when hundreds of genes are identified as differentially expressed, or when differentially expressed genes have an unknown function. In addition, measures of the same gene with different technologies (such as RNA-seq and microarrays) may present a reduced correlation  1  , resulting in different genes detected as differentially expressed  2. Combining gene expression measurements into gene set activity scores (GSAS) have shown to address these critical issues  3, 4. To perform gene expression analyses, GSAS should have three important properties: (1)   representativeness -GSAS should properly encode the transcriptional variance of the dataset; (2)   robustness -biological insights provided by the GSAS should not change if GSAS are recomputed, and GSAS computed on similar samples using different technologies should be highly correlated; and (3) interpretability -researchers should be able to know which genes have higher importance in GSAS calculation.
Methods to project individual gene expression values into GSAS either prioritize their robustness and interpretability or their representativity. Methods that prioritize robustness and interpretability employ a weighted sum of the expression of the genes within the gene set. These weights can be uniform for all genes  5  , or assign positive and negative weights to genes based on the literature  6, 7. Fixing the gene weights ensures that GSAS definition remains consistent across datasets and the contributing genes are known. However, these methods lack representativity, as the gene weights are not proportional to the genes' significance within the gene set. Alternative methods have been developed to model GSAS so they effectively capture the variability of the transcriptome. These methods include utilizing gene ranks  8, 9  , maximizing the variability of genes within a gene set  10  ,.
NetActivity framework and main model We have developed NetActivity, a framework to define sample-wise gene set activity scores (GSAS) from gene expression data. NetActivity consists of an autoencoder-like neural network (NN) that learns a low-dimensional representation of the gene expression data. The encoder maps the input gene expression vector ‫ݔ‬ ‫א‬ ܴ onto a ‫-‬dimensional embedded representation (with ݊ ‫ب‬ ‫. )‬ Each neuron on the embedded space represents a known biological gene set, so each neuron is only connected to the genes of this gene set (Figure  1A ). GSAS are defined as the output of the embedded code before the activation function. Finally, the decoder is composed of a fully connected layer that aims to reconstruct the input gene expression vector from the GSAS (Figure  1A ).
NetActivity's architecture ensures that GSAS are interpretable, while our three-step training ensures that GSAS are representative and robust. In step 1, we initialize the encoder layer of NetActivity by training an autoencoder for each gene set, with the genes in the gene set as input features and one neuron in the embedding layer (Figure  1B ). This step learns the weights so that the output of the embedding layer maximizes the representativity of the expression of the genes in the gene set. In step 2, we freeze the weights from the encoder layer and train the decoder. This step ensures that GSAS are independent of the initialization parameters and hence, robust, by ensuring a smooth training, as done in transfer learning  23. In step 3, we unfreeze all the weights and fine-tune the network by training for additional epochs (Figure  1B ).
At the end of the training, the set of all GSAS will accurately encode the input gene expression, hence optimizing the representativity of the whole transcriptional variability. NetActivity's main model had one layer on both the encoder and decoder. Our main model contained a selection of 1,518 well-known biological gene sets (Sup Table  1 ): 1,485 (Gene Ontology) Biological Processes  17, 18  (BP) (Sup Table  2 ) and 33 KEGG  [19] [20] [21]  Pathways (Sup Table  3 ). The model was trained using all GTEx project 22 samples (9,662), so the model defined GSAS representative of multiple tissues and biological functions (Figure  1C ). This selection of gene sets in combination with the three-step training returned highly robust GSAS, i. e. independent of the weights' initialization (Figure  1D ).
In contrast, including all GO terms and KEGG pathways reduced the GSAS robustness (Sup Fig 1). While NetActivity enables adding additional hidden layers or dropout, none of these configurations significantly improved the performance to compensate for the increased complexity and reduced interpretability of the model (Sup Figs  2 3 4 5 ). Finally, this model is openly distributed in a Bioconductor package (www. bioconductor. org/packages/NetActivity) to facilitate the computation of GSAS on new datasets (Figure  1C ). The gene set encoder layer encodes information from genes to gene sets. Each gene set neuron only receives information from the genes present in the gene set. The decoder layer aims to reconstruct the original gene expression matrix by combining the information from all gene sets via a fully connected layer.
NetActivity framework enables to include multiple layers in the gene set encoder and decoder. B: NetActivity three-step training. In step 1, an autoencoder is trained for each gene set. In step 2, weights from the encoders of step 1 are used to initialize the gene set encoder layer. The weights of this layer are frozen (represented by dashed lines), while the decoder is trained. In step 3, the whole network is trained. GSAS: Gene Set Activity Scores. C: Definition of the main model. The main model is trained using GTEx data and includes GO (Gene Ontology) Biological Processes (BP) and KEGG Pathways as gene sets. The trained gene set encoder is used to compute GSAS in other datasets. D: Robustness of model training. Robustness measures how robust each GSAS is to parameters initialization. A 0 means that the GSAS completely depends on the parameter initialization while a 1 means that is completely independent.
We evaluated the robustness of three training processes: Step 3, i. e. , directly training the whole autoencoder; Step 1 + Step 3, i. e. , training the whole autoencoder after initializing the parameters by running an autoencoder in each gene set; Step 1 + Step 2 + Step 3, i. e. , using the three-steps training. E-F: Biological relevance of NetActivity weights. NetActivity gene relevance estimates the importance of a gene for defining GSAS. The higher the gene relevance, the more gene sets a gene has a strong influence on. E: Correlation between NetActivity gene relevance and number of diseases associated with a gene. Genes are grouped by the number of gene sets they belong to (1, 2, 3-5, 5+). X-axis represents the number of diseases a given gene is associated with in the DisGeNET database  24.
F: Correlation between NetActivity gene relevance and loss of function intolerance (pLI). Gene Sets per gene: number of gene sets where a gene is included. Gene constraint: a measure of the intolerance of a gene to loss of function mutations. Gene constraint categories are computed based on pLI scores from gnomAD: low (<0. 1), medium (0. 1-0. 9), and high (>0. 9). A higher pLI means a stronger depletion of loss of function mutations and higher biological relevance. G: Representativity of the GSAS. Representativity measures the proportion of variance of the original gene expression matrix contained in the whole GSAS matrix. Representativity was measured in the dataset used to train the data (GTEx) and in an external dataset (TCGA). H-I: correlation between the top 10 principal components (PCs) of the gene expression matrix and the GSAS matrix in an IBD cohort (H) and the tumor cohort PRAD from TCGA (I).
NetActivity's three-tier training produces interpretable and biologically consistent GSAS We evaluated whether NetActivity produced biologically relevant and interpretable GSAS. NetActivity gene relevance was defined as the sum of the genes' weights magnitudes across multiple gene sets (see Methods). We compared the inferred NetActivity gene relevance with two measures of biological relevance: (1) the number of diseases associated with a gene in DisGeNET, a database of gene-disease associations  25  ; and (2) the intolerance to loss of function mutations (pLI) from gnomAD  26. Genes with a high intolerance to loss of function mutations are likely to be essential for normal biological function. Genes with higher NetActivity relevance were associated with more diseases (Figure  1E , p-value < 2e-16) and were more intolerant to loss of function mutation (Figure  1F , p-value = 1.
Therefore, step 3 increased the weights of genes relevant to the gene set function, while reducing the weights of irrelevant genes, although they contribute to the gene set variance. GSAS from NetActivity were also representative of the whole transcriptome, retaining most of the transcriptional variance of the input samples, both in the training dataset (GTEx) and in a new dataset of cancer samples (TCGA) (Figure  1G ). Further, principal components (PCs) of GSAS explained a similar proportion of variance to PCs of the input gene matrix (r = 0. 99), both in an external common disease cohort (IBD cohort) or in each TCGA tumor subtype (cancer). Indeed, the first 10 PCs of the original gene matrix were highly correlated with one of the first 10 PCs of the GSAS matrix in both cases (Figure  1H-I ).
NetActivity, unlike previous methods, yields robust GSAS across different technologies and datasets We compared the robustness of NetActivity GSAS with two state-of-the-art methods to encode GSAS: GSVA 9 and Hipathia  13. First, we evaluated the consistency of the GSAS definition across datasets, defined as the correlation between the GSAS and the expression of the genes in the gene set (see Methods). Thus, a method would return consistent GSAS definitions if the GSAS computed in different datasets are correlated with the same genes. We explored the consistency of the GSAS definition in: (1) subsamples in a dataset; (2) datasets generated with different technologies; and (3) datasets from different tissues. We computed GSAS using all TCGA-PRAD samples (prostate cancer project from TCGA) or only the subset of healthy tissue samples, and compared the GSAS of healthy samples in both approaches.
NetActivity and Hipathia returned equivalent GSAS (i. e. GSAS whose Pearson correlation > 0. 9) and consistent GSAS definitions for more than 90% of the gene sets (Figure  2A ). In contrast, only 71% of GSVA GSAS had equivalent values and 44% consistent definitions. Next, we compared GSAS definitions in TCGA-PRAD with those obtained in a dataset from: (1) the same tissue (prostate cancer) but generated with another technique (gene expression microarray samples from GSE169038, GEO-PRAD); or (2) a different tissue (breast cancer, TCGA-BRCA) generated with the same technique (RNA-seq). In both cases, NetActivity returned the most consistent GSAS definitions (Figure  2B ), having the highest proportion of gene sets with a highly consistent definition. In GSVA most of the gene sets had different GSAS definitions, either when comparing different technologies or different tissues.
Second, we evaluated the biological replicability of the GSAS. We first performed a differential expression analysis between samples with low or high Gleason score, a histologic measure of tumor progression, in: (1) PRAD samples from TCGA, generated with RNA-seq (TCGA-PRAD); and (2) prostate cancer samples from GSE169038, generated with a gene expression array (GEO-PRAD). We evaluated the concordance of the results when using single gene expression analysis or the gene set projection methods (NetActivity, GSVA, and Hipathia) (Figure  2C-F ). Hipathia presented the lowest reproducibility (Figure  2E ), even with gene sets significant in both datasets but in opposite directions; while NetActivity and GSVA had the most replicable results (Figure  2D  and 2F ). For both methods, all gene sets differentially expressed in both datasets had a concordant direction.
However, gene sets differentially expressed in GSVA had a modest overlap with the gene sets differentially expressed with NetActivity (Sup Fig 11) because GSVA was limited to gene sets where genes had coherent differences between the groups (Sup Fig 12). Notice that differential expression using individual gene expression measurements was noisier, with some genes presenting large gene expression differences in one dataset but not in the other (Figure  2C ), probably due to technological differences. D: Differential expression of GSVA scores. E: Differential expression of hipathia pathways. Axes report the U statistic of the Wilcoxon test. F: Differential expression of NetActivity GSAS. LogFC is the differential expression estimate from limma. Gene sets (or genes) were colored by their statistical significance in the datasets: Both (green) -features with different expression in both datasets (FDR < 0.
05); TCGA (yellow) -features with different expression in TCGA-PRAD (FDR < 0. 05) but not in GEO-PRAD (FDR > 0. 05); GEO (blue) -features with different expression in GEO-PRAD (FDR < 0. 05) but not TCGA-PRAD (FDR > 0. 05); None (grey) -features without differences of expression in any of the datasets (FDR > 0. 05).
NetActivity enables performing an interpretable gene set metaanalysis combining multiple gene expression platforms We run a GSAS meta-analysis on differences between Gleason low and Gleason high prostate cancer samples. We included nine prostate cancer datasets: three generated with RNA-Seq, three with Illumina HumanHT-12 array, and three with the Affymetrix HuEx1. 0 array (Sup Table  4 ). 152 gene sets had differential GSAS between Gleason low and high samples (FDR < 0. 05). Several of them were associated with cell division and morphogenesis (Sup Table  5 ), coherent with differences in tumor progression between Gleason low and high. The magnitude of the GSAS was comparable between the different technologies (Figure  3 ), which is essential to enable a meta-analysis. Further, we explored the two top gene sets associated with Gleason scores: GO:0048012 (hepatocyte growth factor receptor signaling pathway) and GO:0051984 (regulation of chromosome segregation).
NetActivity advances the knowledge of prostate cancer We applied NetActivity to the PROMOTE study  29, 30  to show its performance in a clinical dataset. The PROMOTE study consists of bone metastatic samples from metastatic castration-resistant prostate cancer patients treated with abiraterone. We characterized whether differences in gene expression were associated with a better response to abiraterone (higher Time To Treatment Change or TTTC), using NetActivity and a traditional differential expression analysis. 30 gene sets had GSAS associated with response to treatment (FDR < 0. 05), with top gene sets related to cell division and cell cycle processes (Sup Table  6 ). Interestingly, 19 top gene sets associated with response to treatment were also associated with tumor progression in the prostate cancer meta-analysis, suggesting that abiraterone treatment reverts cancer progression.
Differential gene expression analyses revealed 592 genes differentially expressed (FDR < 0. 05 and log2 fold-change > 1, Sup Table  7 ) and 248 enriched biological process GO terms (Sup Table  8 , FDR < 0. 05). In contrast, most of the top enriched terms were related to muscle tissue, hence irrelevant for the phenotype, while only a few were related to cell cycle and mitosis (Sup Fig 13). Even when restricting the analysis to genes and GO terms present in NetActivity (Sup Table  9 ), a high proportion of enriched terms were still related to muscle tissue. NetActivity GSAS correlated better with response to treatment than individual genes. For instance, GSAS of the top significant gene set, regulation of attachment of spindle microtubules to kinetochore (GO:0051988), had a stronger association with TTTC (R 2 = 0.
41) than any of these genes individually (Sup Figure  14 ). In this gene set, samples with a better response (higher TTTC) had lower GSAS (Figure  4A ). NetActivity assigned positive weights to the top three genes of this gene set (KNSTRN, ECT2, and RCC2) (Figure  4B ), so these genes also had a negative correlation with TTTC (Figure  4C ). Nonetheless, KNSTRN, ECT2, and RCC2 were not significantly associated with response to treatment after accounting for multiple testing, so these genes were not identified in the traditional gene expression analysis. Therefore, NetActivity can uncover relevant differences in key gene sets which might be missed in a traditional gene expression analysis.
Discussion In this work, we propose NetActivity, a computational framework to define gene set activity scores (GSAS) from individual gene expression values using a sparse autoencoder-like neural network. NetActivity GSAS addresses three long-sought features in GSAS: they are representative of the input gene expression, robust to different initializations and technological biases, and interpretable such that biological insights can be inferred from the weights assigned to genes. Specifically, NetActivity first learns the best representation of each gene set in a reference dataset, assigning a weight for each gene in a gene set. Thus, NetActivity can accommodate gene sets with genes operating in opposite directions and give less weight to less relevant genes. In contrast, some gene set project methods, such as z-score 5 or singscore  7  , assign the same weight to each gene in a gene set and are designed to detect gene sets with genes expressed in the same direction.
Second, NetActivity defines robust GSAS, as the same weights are used to infer GSAS in new datasets. In contrast, gene set projection methods, such as z-score, ssGSEA 8 , GSVA  9  , iPAS  33  , PLAGE  10  , or Pathifier  34  , define gene weights based on the data, so the GSAS definition depends on the dataset where they are created. Thus, a high GSAS can be associated with high expression of a different set of genes depending on the dataset. Indeed, as shown for GSVA, the GSAS of a sample might depend on the samples included in the score computation. Third, although NetActivity starts by defining GSAS that maximize the variability of the genes in each gene set, the model can learn a gene set representation more representative of the gene set biology, especially when the gene set might not be accurately defined, as in the KEGG pathway hsa00430.