dataesr/scientific-paragraphs-categorization huggingface dataset:
Application In this study we worked with gene expression profiles (GEP) issued from myeloma cells (MC) of 602 MM patients and from normal plasma cells (NPC) of 9 healthy donors used in a previous study  [25]. For each GEP, we identified the over/under-expressed genes by comparison to NPC mean expression with a 1. 2-fold. We choose this discretization threshold since it gives the best precision accuracy (lower than 2. 2e-16) when making cross-validation tests with the MM GEPs (data not shown) using the sign-consistency approach described in  [18]. Then, we use the PID-NCI database  [22]  to generate a graph by extracting the downstream events from three  signaling pathways (IL6/IL6-R, IGF1/IGF1-R and CD40)  [26]  to the differentially expressed genes. The obtained subgraph from NCI-PID 2012, contained 2269 nodes, 2683 edges and connected 529 differentially expressed genes (Fig.
Components validation Due to the fact that only two components are composed of more than one gene, we will focus mainly on those components (Table  4 ). For each gene expression profile n and each selected component c, we computed the maximal  In order to validate the similarity computing, we generated for each dataset, 5 randomized datasets by scrambling observed signs. As previously, for each randomized dataset, we computed the MS with the components configuration. Then, for each component, we compared the MS between real data and randomized data with a Welch's t-test (Table  4 , Validation p-value). Both components have a p-value lower than 0. 05, allowing us to conclude to a statistical significance.
Comparison with other clustering methods In order to validate the components identification with our method, we compared it to two graph clustering algorithms: ClusterONE  [12]  and the Cytoscape plug-in ClusterMaker  [30] , which is a fuzzy c-means clustering algorithm. Both methods are similar to the one we propose since they do not include the GEPs for the clustering. We applied both methods with different parameters to the same regulatory network from the PID-NCI database used in our study. To estimate the quality of a clustering method, we consider a good cluster as one that is enriched with specific GO-terms based on the level of the GO-terms in the GO hierarchy. The higher the annotation level of the GO-term, the more specific its annotation will be. For each GO-term present in the ontology we computed its minimal depth from the root term (biological_process : GO:0008150); we found that the mean depth of all GO-terms was 7.
By comparison, when using no clustering method the SE of the full database is of 0. 11. The Loss information ratio column in Table  7  shows a comparison with respect to the case where the full database was used to find specific terms. We compute this number for each clustering method c as 1 -CQ c /CQ PID. This shows that our method is the one that obtains a higher proportion of quality score when compared to the full PID-NCI knowledge, and therefore a lower loss information ratio when compared to the 4 other clusterings. Finally, the closest clustering method (FA 1 ) is based on overlapping genes which can lead to an overestimation of the CQ due to the fact that a gene associated with both clusters will be counted two times.
Our method seems efficient to identify and select functional components specific to the gene expression profiles used in our study taking into account the computational complexity that represents analyzing large-scale networks. However in this case study the reduction to 15 components, with two validated ones with respect to shuffled data, does not allow us to provide a deeper understanding, especially with respect to the subtypes of patients based on the overall survival. As a perspective of this work, we wish to improve the subcomponent identification in order to be able to compute larger regulatory networks, and potentially full databases. For this purpose, we would like to implement the components identification in ASP. Another research line will be to apply this method to other data (regulatory network and observations data) as well as to model with other more refined modeling frameworks the subcomponent C 2 to investigate the patient subtypes overall survival.
Acknowledgements Tis study was supported by  Intergroupe Francophone du Myélome  and by a  French Institute National du Cancer  Grant  EVACAMM PROG/09/10  (to H. A. L. , S. M. ), a  National Institutes of Health  Grant  PO1CA155258-01  (to S. M. , H. A. L. , N. C. M. ), and a research grant from  Celgene. B. M. 's PhD scholarship was funded by  GRIOTE  project. We would like to thank  Elise Douillard ,  Magali Devic ,  Emilie Morenton  and  Nathalie Roi  for excellent technical assistance. We are most grateful to the  bioinformatics core facility of Nantes (BiRD -Biogenouest)  for its technical support.
Genome-scale metabolic models have become the tool of choice for the global analysis of microorganism metabolism, and their reconstruction has attained high standards of quality and reliability. Improvements in this area have been accompanied by the development of some major platforms and databases, and an explosion of individual bioinformatics methods. Consequently, many recent models result from "à la carte" pipelines, combining the use of platforms, individual tools and biological expertise to enhance the quality of the reconstruction. Although very useful, introducing heterogeneous tools, that hardly interact with each other, causes loss of traceability and reproducibility in the reconstruction process. This represents a real obstacle, especially when considering less studied species whose metabolic reconstruction can greatly benefit from the comparison to good quality models of related organisms.
This work proposes an adaptable workspace, AuReMe, for sustainable reconstructions or improvements of genome-scale metabolic models involving personalized pipelines. At each step, relevant information related to the modifications brought to the model by a method is stored. This ensures that the process is reproducible and documented regardless of the combination of tools used. Additionally, the workspace establishes a way to browse metabolic models and their metadata through the automatic generation of ad-hoc local wikis dedicated to monitoring and facilitating the process of reconstruction. AuReMe supports exploration and semantic query based on RDF databases. We illustrate how this workspace allowed handling, in an integrated way, the metabolic reconstructions of non-model organisms such as an extremophile bacterium or eukaryote algae.
To accomplish such applications successfully, an effort to correctly incorporate all the relevant available information must be made first, so that the quality of the reconstructed GSM is the best possible. To this end, a well-described protocol for generating high-quality GSMs has been made available  [6]. Additionally, there are several databases such as KEGG  [7] , BioCyc  [8] , BiGG  [9]  or Model SEED  [10]  that aggregate metabolic data on which GSMs can be built  [11, 12]. For instance, the contents of the MetaCyc database have been curated from 54,000 articles  [8]. Moreover, many independent methods have been developed to generate GSMs, mostly based on the aforementioned databases, including some toolboxes and workspaces. These latter allow a user to chain several tools into pipelines and have proven their efficiency in building high-quality GSMs.
Among them are Pathway Tools  [13] , the Raven Toolbox  [14]  and The SEED  [10]. Pathway Tools was the first platform connecting biological metadata by https://pypi. python. org/pypi/padmet and Gitlab https://gitlab. inria. fr/maite/padmet. PADMet-utils can be downloaded on Gitlab https://gitlab. inria. fr/ maite/padmet-utils and used in stand-alone mode. MeneTools Python package is available on https:// pypi. python. org/pypi/MeneTools. An ad-hoc license was set up for the integration and operation of these different tools, reserving the use of the software to academic users only. The exploitation of any results generated by the workspace for commercial purposes is prohibited. relying on the BioCyc database via their own internal format. It also ensures traceability and reproducibility for the tools implemented in the platform.
The Raven Toolbox integrated multiple data source types to encompass the variability of available data in reconstruction processes. The SEED proposed a complete automatic generation of models for prokaryotes and plants. There are also online workspaces such as Galaxy  [15]  or KBase  [16]  that enable the creation and customization of pipelines, while exploiting their own intrinsic databases. Most of these platforms internally trace the source of every reaction and metabolite in a GSM, that we call process metadata. For instance, Pathway Tools supports the use of evidence codes, citations, and user comments to document the origin and reason why information is included in a model. This information allows reports to be produced for comparing different versions of a model provided that the model construction is completely built within a single platform.
Thiele and Palsson's protocol steps are generally followed during reconstructions although customized to the availability of data sources and tools. As an example, the study of nonmodel organisms may involve comparisons to genomes and metabolisms of several taxonomically-related organisms. In such cases, the output of a main platform requires adjustments assisted by a choice of specialized tools. In this sense, as illustrated in Table  1  (see Results), many recent GSMs were obtained by combining a major platform with additional methods relying on several databases, leading to "à la carte" reconstruction pipelines. The output of such personalized pipelines is exported in standard formats such as SBML or stoichiometric matrices, leaving out information about sources of reactions (e. g. , the method and reason why a reaction was added to a model) and process metadata which cannot be recovered with versioning systems.
The reproducibility of these personalized reconstructions is threatened by the lack of metadata availability caused by the use of multiple methods and toolboxes in "à la carte" pipelines. Hence, tracking of process metadata is needed when using tools that accomplish dedicated subtasks of metabolic model reconstruction without being part of an existing platform. This would allow transparency throughout the reconstruction process, as discussed by Heavner and Price  [17]. To circumvent these issues, we present the workspace AuReMe (AUtomatic REconstruction of MEtabolic models). AuReMe is designed to house "à la carte" reconstructions and analysis of GSMs while ensuring that their metadata is properly stored and can be efficiently explored and distributed. In particular, it can complement reconstructions provided by external platforms, such as Pathway Tools for which the AuReMe import preserves the existing process metadata.
Heterogeneity of reconstruction processes reveals heterogeneity in traceability and metadata availability We surveyed the reconstruction procedures of 19 published GSMs listed in Table  1. These GSMs were selected because they cover main phylogenetic branches including eukaryotes (ascomycete yeasts, green and brown algae, terrestrial plants and human), eubacteria (cyanobacteria, proteobacteria, spirochaetes and firmicutes) and euryarcheote archea. GSMs were selected to include highly studied organisms such as Saccharomyces cerevisiae and non-model species as well, such as Acidithiobacillus ferrooxidans. Thus, we avoid bias related to the level of information available for the reconstructed organisms or its phylogenetical cluster. We compared these GSMs in terms of metabolic model content, selected databases, available metadata and reconstruction processes.
We first observed that most models display "biological" metadata, i. e. , metadata related to the model itself (gene associations, external references, etc. ) and to its connection to other resources of knowledge (template metabolic networks, protein or chemical databases, etc. ). This information is currently provided in SBML files, the most widespread export format. Then we compared these GSMs in terms of selected databases and reconstruction processes. We observed that annotation, use of one or several (up to 15) template models, gap-filling and manual curation are four widely shared steps, which are consistent with the general methodology described by Thiele and Palsson  [6]. However, we also noticed that different tools and methods were used in the reconstruction, which confirms the hypothesis of "à la carte" reconstruction pipelines.
In addition, we noted that several databases of reactions were usually used for reconstructing models (presence of identifiers related to the main databases: KEGG, BiGG, BioCyc, Model SEED). In fact, one database and one method of reconstruction is rarely enough to obtain a model. For example, Bosi et al  [24] , included information from all the aforementioned databases to reconstruct 64 models of Staphylococcus aureus. Notice however that use of multiple reaction database requires a consolidated curation procedure to avoid duplicate reactions  [25]. Starting from these observations, we investigated further the possibility of tracing the origin of reactions. This is a main part of what we define as "process" metadata (Supp. Fig A in S1 File), related to the reconstruction processes: steps at which reactions were added (automatic reconstruction, gap-filling, manual curation), and the information sources they depend on (annotation, orthology, etc.
). These metadata make it possible to i) locate and connect the studied model along with other models and knowledge resources and ii) trace the reconstruction processes and ensure their reproducibility. Our study of the 19 GSMs highlighted that, when available, the process metadata of reactions were provided on multiple supports that were often neither machine-readable (pdf files, Excel files, notes in SBML files) nor suitable for further exploration. There was often no means to decipher at which step of the reconstruction and the reason why a particular reaction was added, making reproducibility of the model generation more difficult. In particular, manual curation was not always explicit. The only evidence that this process had been conducted was the presence of reactions with unreferenced identifiers, which do not match identifiers in the database(s) described as being used for the reconstruction process  [26].
A package and a workspace to personalize and trace GSM reconstruction As described previously, current methods allow the reconstruction of high-quality GSMs but do not always take into account the need for metadata storage and exploitation to facilitate the study and reproducibility of models. We designed a unified workspace AuReMe (AUtomated REconstruction of MEtabolic models) to house the "à la carte" reconstruction of GSMs (Fig  1 ). This workflow controller, based on the Docker technology is the conductor handling the order of methods used in personalized pipelines. It is associated with a local data manager, the Python package PADMet (Python library for hAndling metaData of METabolism), whose role is to store information related to the sequence of tools used in the pipeline. Finally, PADMetutils encompasses several tools and methods to curate, analyze a GSM considering topological or flux modeling, generate wikis, produce reports and export the models.
AuReMe gathers academic-free tools and enables the design of reconstruction pipelines that are flexible and can suit various available data sources (genome annotation, template GSMs, protein sequences, etc. ) while storing metadata to ensure reproducibility of reconstructions (Supp. Fig A in S1 File). It can follow four major steps of reconstruction processes: annotation or orthology-based modelings, gap-filling and manual curation. In addition, AuReMe supports most processes of the Thiele and Palsson protocol  [6]  by proposing tools and methods that facilitate analysis and storing of the results at each step related to experiments or exploration of literature. In particular, the refinements to reconstruction are strongly related to the management of metadata performed in AuReMe. Manual curation is assisted and formalized within forms to be filled before being integrated in the pipeline treatment.
Additionally, analysis tools based on flux or topology are also included in the workspace. Contrary to existing platforms, AuReMe works with three major databases that are freely or academic-freely available: BiGG, ModelSEED and MetaCyc, for which some versions are already included. AuReMe also works with the KEGG database provided that the user has the appropriate licence. The use of those databases facilitates the open-data initiative, that our workspace wants to promote. At any time during the reconstruction process, the visualization of model data and associated metadata is available through the generation of a local wiki, that can be connected to the Meta-Cyc, the BiGG or theSEED database used for reconstruction. Each run of AuReMe requires to select a main reference database but reactions from other databases (such as those predicted with orthology-based methods from models using alternative database identifiers) can be inserted in the reference database after a mapping operation based on the MetaNetX dictionary  [45].
Outputs of AuReMe can be self-sufficient or be integrated again in many existing platforms. https://doi. org/10. 1371/journal. pcbi. 1006146. g001 The first feature of the AuReMe workspace is its adaptability to various input data and databases. The PADMet package format ensures the interoperability of knowledge, tools and data (Fig 1 ). A wide range of input data types and the three pre-set databases (see S1 File for details) enable the exploitation of all genomic and metabolic exploration within the workspace. The second feature of the AuReMe workspace is the customization of a pre-set pipeline. For example, the result of an annotation-based reconstruction can be imported into the workspace with the purpose of being merged with one or several orthology-based network(s), or other pre-existing models.
As done in KBase and Pathway Tools for instance, manual curation (creation, modification and deletion of metabolites/reactions) is assisted via the use of forms. All manual update operations are stored internally by the PADMet data-manager. This allows a user both to trace the reasons for adding the reactions and to automatically include ongoing manual curations in a future version of the model. The purpose is to ensure the consistency and sustainability of metadata, especially when the pipeline has to be run again, due to the availability of a better version of input data (new genome assembly for instance), of updated version of databases or of new version of tools included in the customized pipelines. The last feature of the AuReMe workspace is to be opened and complementary to other platforms in order to facilitate further analyses.
Wiki-based exploration of metabolic networks: A novel method to explore and monitor GSM reconstructions and their associated metadata In addition to the easy connection to graph-visualization tools such as Cytoscape  [48] , PAD-Met-utils proposes several solutions for exploring GSMs such as the creation of pathway-completeness text reports or graphic reports. As a main originality, a local wiki containing all the information related to the model, including its process metadata and links to external online databases (See Supp. Fig A in S1 File) can be created. For the sake of traceability, we favoured the use of assisted manual curation forms to perform GSM updates. Therefore, the wiki cannot be edited. Its main advantage is to provide a multi-page browsable exploration of thousands of heterogeneous entities related to a GSM together with a semantic search module.
This convenient and user-friendly view based on linked data allows to concentrate and trace all the information of the GSM components and the reasons why they were introduced in the network, and also widens it to external information on the web. As depicted in Fig  2 , the user can browse the contents of the GSM starting either from reactions, genes, metabolites and pathways (when available, which is the case for models relying on Metacyc database) or from the methods used to create the network. In the E. siliculosus example, the GSM contained 1977 reactions in total: 1661 were recovered from genome annotations, 440 were deduced from orthology-based tools, 85 were added by gap-filling tools and 65 were manually corrected in order to fill biologically-relevant pathways which had a few missing reactions, according to the pathway completeness-rate, after annotation and orthologybased procedures.
At each step, we gathered information about the model (Supp. Table A in S1 File). Main reconstruction steps for each organism included annotation and/or orthology, merging of models, gap-filling and/or manual curation. Templates for orthology were selected either i) for being the best curated GSMs for organisms that are models in a taxonomic rank of the studied organism, and or ii) for being well-curated models available for taxonomically close organisms. For instance, for S. thermosulfooxidans str. Cutipay, Clostridium ljungdahlii was chosen as a template because it is the phylogenetically closest microorganism in BIGG database. Although C. ljungdahlii is the closest microorganism, it is anaerobic. As S. thermosulfooxidans is aerobic, Bacillus subtilis was chosen as a representative for aerobic microorganisms and also because of its high-quality published model.
Added-value of the pipeline procedure for pathway completion: Case-study on E. siliculosus tetrahydrofolate biosynthesis The benefit of tracking process metadata during the combination of orthology, annotation and gap-filling is also noticeable at the pathway scale. Complementary methods that exploit all available data can retrieve several reactions from pathways, resulting in the reconstruction of complete or near exhaustive pathways. The wiki page associated with a given pathway describes all the methods of the pipeline providing (multiple)-evidences for the presence of each pathway reaction in the considered species GSMs and allows browsing databases to search for new possible evidences with other species. An example of such pathway completion can be observed during the reconstruction process of E.
As mentioned on the MetaCyc website, the database groups reactions and metabolites into classes using an ontology tree structure. In our example, a 7,8 dihydrofolate (DIHYDROFO-LATE-GLU-N) is a metabolite class comprising subclasses and instances. The 7,8-dihydrofolate monoglutamate (DIHYDROFOLATE) compound is one of these instances. Originally, reaction DIHYDROFOLATESYNTH-RXN produces DIHYDROFOLATE while the following ) that takes DIHYDROFOLATE as a reactant. This enables to restore the producibility of the final compounds of the pathway starting from its inputs. A second reaction is added by gap-filling in PWY-6147. Monitoring with the wiki and the various reports are helpful to keep track of this complex reconstruction process. Application of those heterogeneous methods allows the completion of the entire dihydrofolate biosynthesis pathway from GTP as described in the MetaCyc database.