dataesr/scientific-paragraphs-categorization huggingface dataset:
g. co-regulation, co-expression or membership of a protein complex, of a metabolic or signalling pathway or of a cellular aggregate (e. g. chaperone, ribosome, protein transport facilitator, etc. )'. We used GO Biological Processes as prototypic FMs and computed two topological descriptors  (28). The average mutual rank was not predictive of a module class, random or functional; in contrast, the average degree largely discriminated the two classes of modules (Supplementary Figure  S1 ). In the present work, we generalized this approach and performed a systematic study of modules topological descriptors. We used LASSO (least abso-lute shrinkage and selection operator)  (29)  and LDA (linear discriminant analysis)  (30)  to find a linear combination of the relevant descriptors that optimally separated FMs and RMs.
Topological descriptors and topological score Topological descriptors, also named 'topological parameters', of a network are metrics that characterize a facet of a network topology such as the density of edges, the length of the shortest path (hop count) between two nodes, the density of edges between the neighbours of a given gene. (34) (35) (36). Selecting the relevant topological descriptors. To construct the database to train the machine learning model, we generated all, i. e. 978, GO-BPs (FMs) that comprised 20-500 genes; we also generated 1000 RMs that contained 20-500 randomly sampled genes and whose size distribution is comparable to that of the GO-BPs (Supplementary Figure  S3 , Size). We computed 12 descriptors (definitions are given in Supplementary File, section 3) from the 1978 modules and summarized the results in Supplementary Table  S1.
We had a set of modules D = (x 1 , y 1 ),. , (x 1,978 , y 1,978 ) such that for every module i, the vector x i = (x i 1,. , x i 12) comprised the 12 descriptors of module i. The vector of binary responses y i informed about the module class, 'FM' or 'RM'. The logistic regression model assumed that P ( y i = FM|x i ) = 1 1 + e -x i α , ( 2 ) where α ∈ R 12 was the vector of the regression coefficients. We used this matrix to learn a LASSO model and select the predictor variables (relevant descriptors). The LASSO estimated the regression coefficients by minimizing the least squares quadratic criterion under a constraint on the sum of the absolute values of the coefficients or, equivalently, by minimizing the quadratic criterion penalized by the norm L1 of the coefficients. This technique was aimed at removing the redundant or irrelevant descriptors without much loss of information.
The LASSO method selected 6 relevant descriptors: Shortest Path, Betweenness Centrality, Average Degree, Inverse Centrality, Clustering Coefficient, and Size. The distribution of each relevant descriptor for RMs and FMs is shown in Supplementary Figure  S3. As expected, we observed that the average Betweenness Centrality, the Average Degree and the average Clustering Coefficient were higher in FMs than in RMs, confirming that FMs tended to be denser and more compact than RMs. Defining the topological score of a gene set. We then applied LDA to find a linear combination of the relevant descriptors that optimally discriminated FMs and RMs  (37). We extracted a training set from Supplementary Table S1 by keeping the columns for the 6 relevant descriptors and the response vector. As two classes were to be predicted, only one discriminant axis could be built; we denoted Score Topo the coordinate of each module on this axis.
In Figure  2B , the histogram obtained through cross-validation showed that all values were above Figure  1. The distribution of Score Topo for random and functional modules. The topological score is a linear combination of the 6 descriptors of module topology selected by the LASSO method, i. e. shortest path, betweenness centrality, average degree, inverse centrality, clustering coefficient, and size. It was computed for the 978 Gene Ontology Biological Processes consisting of 20-500 genes and for 1,000 random modules of the same sizes. The Score Topo distributions for random (blue) and functional (red) modules indicated that Score Topo largely differentiated both types of modules. We performed linear discriminant analysis of the Score Topo computed for 978 Gene Ontology Biological Processes (FMs, functional modules) consisting of 20-500 genes and 1,000 random modules (RMs) of the same sizes.
, go k }, Sim(go, GO), is then defined as the maximum se-mantic similarity between the term go and any of the terms in set GO. That is Sim(go, G O) = max 1≤i ≤k (S GO (go, go i )) , ( 3 ) where S GO (go, go i ) is the semantic similarity of the two terms go and go i. Given two genes g i and g j annotated by GO term sets GO i = {go i1 , go i2 ,. , go im } and GO j = {go j1 , go j2 ,. , go jn } respectively, Wang and colleagues (31) defined their functional similarity as Sim g i , g j = 1≤k≤m Sim go ik , G O j + 1≤k≤n Sim go jk , G O i m + n (4) We used the geneSim function, with measure = 'Wang', of the GOSemSim package to compute the semantic similarity of two genes. We computed β, a matrix of functional similarity between all the genes in COXPRESdb, based on this method. For any module, denoted M, composed of n genes, we extracted the sub-matrix of β representing the similarity between all pairs of genes in the module.
Implementing the TopoFun genetic algorithm Starting from a gene set, we aimed at discovering novel genes co-expressed with members of that gene set. The number of candidate genes was far too large for a systematic scanning. We therefore implemented the scanning process through a genetic algorithm (GA), which involved the construction of a fitness function. In the GA, M 0 is the original gene set, M 1 is one of the 500 gene sets produced by Topo-Fun at each iteration and M f is the final, ranked #1 gene set after TopoFun converged. Fitness function. In order to rank modules, we designed a fitness function that quantified the quality of these modules with respect to both co-expression and shared functions. The Supplementary File, section 5, describes the different steps that led to the definition of the fitness function FF.
However, their range were very different; the third term obviously belonged to [0,1], and so did Score Fun. In contrast, Score Topo ranged from -2. 535 to +6. 466 on the learning data set (Figures  1 and3). We introduced ␦ 0 (= 0. 281608) and ␦ 1 (= 0. 1110948) to rescale Score Topo using the learning dataset so that the RM with the lowest Score Topo was set to 0 and the FM with highest Score Topo was set to 1. Genetic algorithm. GA are inspired by natural selection  (42) , and are useful in complex optimization problems  (43, 44). Here, we used a GA to identify 'optimized' modules by maximizing FF. We initialized the algorithm with a population constituted of different modules. At each iteration, the population evolved according to three operators described in the next paragraphs: crossover, mutation and selection.
TopoFun gathered known functional relationships To test the effectiveness of TopoFun, we performed a simulated experiment using GO-BPs as benchmark FMs. We left a number of genes out of different GO-BP modules, ran TopoFun on these 'trimmed' modules, and assessed whether the omitted genes were reinstated. We focused on the set of the 978 GO-BPs that comprised 20-500 genes. To assess whether the topology and size of the GO-BPs affected the effectiveness of TopoFun, we partitioned the 978 modules into 16 classes according to the quartiles of the module size and normalized Score Topo values. The number of GO-BPs in each of the 16 classes is displayed in Supplementary Table  S2. We randomly selected five GO-BPs in each of the 16 classes. From each of the 80 selected GO-BPs we generated three sub-modules by keeping 20, 40 or 60% of the genes in the largest clique (LC) and all the genes in the complement of the largest clique (LC C ).
We generated three additional sub-modules by keeping the LC and 20, 40 or 60% of the genes in the LC C. We ran TopoFun on the resulting 480 sub-modules and calculated the percentage of omitted genes that were reinstated by TopoFun in the final module. Genes omitted from the LC were more efficiently reinstated than those from the LC C (Figure  4A ). The percentage of reinstated genes was also more variable for the genes of the LC than for those of the LC C (Figure  4A ), suggesting that genes in the LC were critical in TopoFun. The different behaviours of LC and LC C genes are also illustrated in Figure  4B. The percentage of LC C genes reinstated was highly dependent on the percentage of LC C genes kept; the more LC C genes were kept in the sub-module, the higher the percentage of LC C genes were reinstated (Figure  4B , lower panel).
In this case, the Score Topo value or the number of genes in M 0 only marginally affected the reinstating rate. In contrast, when genes from the LC were omitted, the size of the M 0 module largely affected the reinstating rate, at least for the 20 and 40% LC sub-modules (Figure  4B , upper panel). This was likely due to the fact that M 0 modules with <25 genes had a LC that often comprised as low as five genes (Supplementary Supplementary Figure  S7 ). Keeping 20% of these five genes resulted in just one gene, which made TopoFun much less efficient. Surprisingly, the Score Topo of the M 0 module was not drastically affecting the reinstating rate but, possibly, for the smallest modules (Figure  4B ). In short, these data indicated that TopoFun efficiently discovered novel functionally related genes starting from a functional co-expression module.
TopoFun discovered novel functional relationships To illustrate the performance of TopoFun, we excluded the smallest GO-BPs for reasons mentioned above and focused on the 193 GO-BPs with 50-100 genes (Supplementary Table  S3 ). Because some genes were not in COXPRESdb, the actual gene number ranged from 46 to 99. For each GO-BP, i. e. M 0 , we ran TopoFun and selected a final, 'optimized' module M f. Supplementary Table  S4  displays the number of genes, the Score Topo , and the Score Fun for each M 0 and M f. We calculated the ratio of each variable for M f to M 0 , and plotted the distribution of these ratios in Figure  5. The final modules comprised substantially more genes (Figure  5 , left panel), with improved topology (Figure  5 , middle panel) and functional similarity (Figure  5 , right panel) than the original modules.
This indicated that TopoFun was able to aggregate novel genes to existing GO-BPs while preserving, most often improving, the functional similarity of the constituting genes. We tested whether the novel genes gathered by Topo-Fun were functionally relevant using STRING, 'a database of known and predicted protein-protein interactions, including direct (physical) and indirect (functional) associations'  (49) , unrelated to COXPRESdb. For each GO-BP, we counted the number of edges between the novel genes, i. e. M 0 C = M f -M 0 , and those in the M 0 module. For each M 0 , we randomly sampled 10 000 sets of |M 0 C | STRING IDs and counted the number of edges between each random set and M 0. Using the empirical cumulative distribution function, we calculated the Benjamini-Hochberg-corrected (50) P-value to observe a number of edges as high as that observed for M 0 C.
Of the 193 GO-BPs, 182 had P-values < 1 × 10 -4 , four had P-values < 1 × 10 -2 , one had a P-value ∼0. 012, and six were not significant (Figure  6A , Supplementary Table  S5 ). Figure  6B -F illustrates representative STRING networks obtained with 'optimized' modules generated by TopoFun, starting from GO:000082∼G1/S transition of mitotic cell cycle (Figure  6B ; edge enrichment, 10. 6; P-value < 1 × 10 -4 ), GO:0051865∼protein autoubiquitination (Figure  6C ; edge enrichment, 3. 90; P-value < 1. 10 -4 ), GO:0043401∼steroid hormone mediated signaling pathway (Figure  6D ; edge enrichment, 1. 49; P-value < 1 × 10 -4 ), GO:0010923∼negative regulation of phosphatase activity (Figure  6E ; edge enrichment, 0. 88; P-value, ns). Those modules with a weak edge fold enrichment still proved to be interesting.
TopoFun identified disjoint co-expressed gene sets in a single GO biological process Another interesting, atypical case is GO:0007596∼blood coagulation (Figure  6F ). This GO-BP was the only GO-BP that TopoFun downsized (Figure  5 , left panel) and was one of the six modules for which the number of edges between M 0 C and M 0 was not significantly increased (edge enrichment, 1. 00; P-value = 0. 51). This GO-BP originally comprised 71 genes in COXPRESdb; TopoFun excluded 11 genes and added two novel genes. Gene Ontology defines 'blood coagulation' as 'the sequential process in which the multiple coagulation factors of the blood interact, ultimately resulting in the formation of an insoluble fibrin clot'. Most of the genes in this GO-BP were related to the proteolytic cascade involved in clot formation and its regulation: complement components, coagulation factors, serine proteases and inhibitors thereof, fibrinogens, thrombomodulin, thrombin receptors.
In short, these data suggested that nine of the 11 genes left out by TopoFun were annotated 'blood coagulation' because they function in platelet lysosome formation. We concluded that GO:0007596 comprised (i) genes. Distribution of the size ratio, Score Topo ratio, and Score Fun ratio. We ran TopoFun on 193 GO-BPs comprising 50-100 genes. For each M 0 (= GO-BP) and M f (= 'optimized' module), we determined the number of genes, the Score Topo and the Score Fun , and plotted the distribution of the ratios of these variables for M f to M 0. expressed in the liver that participated in the proteolytic cascade involved in clot formation and (ii) genes expressed in megakaryocytes (the platelet progenitors) that participated in platelet formation and, ultimately, clotting; it is then not surprising that the two categories of genes were not co-expressed.
Because the genes of the first category were more numerous in GO:0007596 than those of the second, the largest clique of this BP was composed exclusively of genes of the first category. Because the genetic algorithm used in TopoFun preserved the largest clique of the initial GO-BP at each iteration, genes of the second category were excluded. In the STRING database, the 11 genes left out and the two genes gathered by TopoFun did not display any interaction with the other genes in GO:0007596 or between them (Figure  6F ).
TopoFun is not biased towards a limited number of highly connected modules Because TopoFun favoured highly connected modules, we were concerned that it may frequently converge towards a limited number of highly connected sets of genes that behaved as 'attractors' during the genetic algorithm iterations. We computed the overlap distance (for two sets S 1 and S 2 , overlap distance(S 1 , S 2 ) = 1 -(|S 1 ∩ S 2 | / min(|S 1 |, |S 2 |)) for each pair of 'optimized' modules generated by Topo-Fun. We used this distance to perform hierarchical clustering of the 'optimized' modules and displayed the results as a heat map (Figure  7 ). The 193 modules were grouped into 38 clusters, of which five included 38, 34, 27, 20 and 17 modules. The Gene Ontology terms are organized in a hierarchical tree where one given term is a subclass of a more general term.
Comparison to classical network algorithms A benchmark study comparing TopoFun to classical network algorithms such as WGCNA  (13)  and CoExpNets  (24) , ARACNE  (25) , GeneMania (26) etc. is highly desirable but practically not possible as TopoFun is gene setcentred. TopoFun could theoretically identify modules systematically, e. g. starting from each gene and its nth-degree neighbours. It is however not practically possible due to the computation time of the GA (one iteration takes about 3. 5 s on a laptop). As it is now, TopoFun is better suited to 'optimize' modules identified by classical network algorithms. Accordingly, we ran WGCNA (with minClustSize set to its default value, 20) on COXPRESdb Mmu-c3-1 and identified 253 modules comprising 20-701 genes (median, 57; iqr, 59). For a fair comparison, we focused, as we did previously with GO-BPs, on the 84 WGCNA modules with 50-100 genes.
We ran TopoFun on these 84 M 0 modules and obtained the corresponding M f modules (Supplementary Table  S7 ). We looked at the distribution of the module size, Score Topo and Score Fun (Supplementary Figure  S8 ). The WGCNA modules displayed very good Score Topo , (0. 727 ± 0. 070) compared to that of RMs (0. 147 ± 0. 067) and GO-BPs (0. 424 ± 0. 105) of the same size. Surprisingly, the Score Fun of the WGCNA modules (0. 246 ± 0. 081) were not very different from those of RMs (0. 207 ± 0. 032) and The green dots correspond to the primary genes of the GO-BPs and the orange ones to the novel genes gathered by TopoFun (B) GO:000082∼G1/S transition of mitotic cell cycle (edge enrichment, 10. 6; P-value < 1 × 10 -4 ), (C) GO:0051865∼protein autoubiquitination (edge enrichment, 3. 90; P-value < 1 × 10 -4 ), (D) GO:0043401∼steroid hormone mediated signalling pathway (edge enrichment, 1.
49; P-value < 1 × 10 -4 ), (E) GO:0010923∼negative regulation of phosphatase activity (edge enrichment, 0. 88; P-value, ns), (F) GO:0007596∼blood coagulation (edge enrichment, 1. 00; P-value, ns). The blue dots correspond to primary genes that were part of GO:0010923 and were excluded by TopoFun (see Results for details). only half of those of GO-BPs (0. 480 ± 0. 070). After running TopoFun on the WGCNA modules with 50-100 genes, we obtained new modules that displayed a modest (1. 09) but significant (P < 0. 01, paired Student's t-test) increased Score Topo (0. 789 ± 0. 200). The size of the modules was substantially increased from 70. 2 ± 13. 7 to 333 ± 32. 3, and so were the Score Fun (0. 752 ± 0. 065). We concluded that the modules produced by WGCNA had an exceptional topology, more compact and dense than most GO-BPs of similar size; however, the constituting genes were only weakly functionally related as their functional similarity was not very different from that of randomly sampled genes.
DISCUSSION We report on TopoFun, a novel machine learning method aimed at discovering genes co-expressed with and functionally related to a seed gene set. TopoFun required two types of input data: a gene co-expression network (GCN) and a gene annotation set. The choice of both potentially influenced its outcome. In the present work, the GCN (COXPRESdb) was constructed from 31 479 Affymetrix microarray data. The corresponding biological samples were very diverse and did not represent a specific cell type, tissue or experimental condition. This particular data set prevented the observation of cell type-or tissue-specific co-expression links as all samples were used for the calculation of a single PCC for any gene pair. Despite this limitation, TopoFun was able to aggregate novel genes with existing GO-BPs.
A single PCC was also used in GeneFriends, a GCN constructed from RNAseq data  (51). Other co-expression databases used PCC in a way that preserved the information about the samples from which the expression data were obtained. For instance, Gemma recorded the number of independent experiments in which two genes were co-expressed, defined as PCC >0. 7 for experiments with 20+ samples (52). This co-expression metrics enables sorting experiments, e. g. those derived from a specific tissue, before constructing the GCN; it is then possible to observe sample type-specific co-expression relationships. Another possibility is to construct an ab initio GCN from expression data of a specific tissue, cell type or biological condition. The recent availability of single cell expression data in particular offers the opportunity to ask questions about gene co-expression in defined cell types.
TopoFun also required an annotation set, namely Gene Ontology Biological Processes, which greatly influenced the outcome of the method. First, the annotation data deter-mined the construction of the database to train the machine learning model. If some GO-BPs comprised genes that were actually not co-expressed with their BP fellows, one may expect that the topology of some 'functional' modules of the training set was not optimal to identify co-expressed genes. Figures  1  and 3  demonstrated that this situation was not uncommon; a number of functional modules displayed average degree (Supplementary Figure  S1 ) or topological scores (Figures 1 and3) similar to those observed in random modules. One possibility was that some BPs were composed of genes that were not co-regulated; the activity of signalling cascades are often regulated by protein phosphorylation rather than transcriptional co-regulation.
The observation of the relative values of Score Topo and Score Fun will help identify gene sets, including GO-BPs, that are transcriptionally co-regulated vs. gene sets that are not. Alternatively, some GO-BPs were composed of genes that contributed to the same biological process at the organismal level, but were expressed in different tissues/cell types and should not be called co-expressed. The latter possibility was illustrated by GO:0007596∼blood coagulation, as described in the Results section. This example illustrated the ability of Topo-Fun to discriminate sets of functionally related genes based on co-expression data, even if they were annotated by the same GO-BP. The above mentioned heterogeneity of some GO-BPs suggests possible improvements of TopoFun. In the present work we considered the whole GO-BP as a functional module; we could be more restrictive and consider only the largest clique of each GO-BP to train the statistical model.
In addition, provided that the RM misclassification rate was low enough, the genetic algorithm could compensate the high FM misclassification rate, as it involved an iterative process, the comparison of numerous modules, and the selection of only the best one. Again, training the model with GO-BP sub-modules rather than the whole GO-BPs should considerably improve the prediction performance of the topological score. To illustrate TopoFun capabilities, we showed how it assigned novel genes to existing GO-BPs. The novel genes were validated using STRING, an independent knowledge database. We also showed that TopoFun was able to add new genes to modules produced by WGCNA. While preserving the module topology, TopoFun increased the functional similarity of the constituting genes, which was found to be minimal in the original modules.
One may argue that, because the aim of network algorithms is precisely to find new modules not previously annotated by a function, Topo-Fun was too conservative and excessively favoured functionally annotated genes. Because classical network algorithms should also be able to find functionally annotated modules, one may expect that a large proportion of the modules identified by these algorithms would be functionally distinguishable, which was not the case. We concluded that the weighted combination of topological and functional information provided by TopoFun allowed to optimize WGCNA modules to more plausible ones. The 'optimized' modules produced by TopoFun were generally larger than the original modules they stemmed from (Figure  5 ). This is a general trend with network algorithms that tend to produce large modules whose biological interpretation is difficult  (12, (53) (54) (55).
Heuristic corrections were proposed to remove the bias towards large modules  (53, 54, 56)  but resulted in limited efficacy  (55). In Topo-Fun, we payed attention to the problem when we designed the fitness function of the GA. The function was composed of three, equally weighted terms. The third term, |M 1 ∩ M 0 | |M 0 | , was neutral with respect to modules size. It favoured modules that contained the largest proportion of the seed genes; when more genes were added, this term did not increase, unless the added genes were in the initial set of genes (these genes may have been removed at previous iterations). The second term, Score Fun , obviously disadvantaged large mod-ules comprising many genes with no functional similarity to the seed genes. Score Fun allowed larger modules only if they consisted of functionally similar genes, preserving the modules interpretability.