dataesr/scientific-paragraphs-categorization huggingface dataset:
Imaging Acoustic Sensor for Sodium Conditions This 2-antenna acoustic system is devoted to sodium viewing with the following applications: global imaging of the large primary circuit (some meters dimension), hypothetical large disorder detection (structure buckling), and potential lost-parts identification. The present paper exhibits the capabilities of such an enhanced imaging sensor which has been optimized with the CIVA software: experimental measurements were carried out in a water tank, and results show a good qualitative agreement between CIVA simulated results and the measures regarding spherical and planar target images. Having in hand this original concept of a 2-antenna acoustic system and the simulation CIVA code validated on commissioning water test results, it has been possible to design a first sodium prototype.
Introduction Biology is becoming data-intensive as high throughput experiments in genomics or metabolomics are rapidly generating datasets of massive volume and complexity  (Marx, 2013; Schadt et al. , 2010) , posing a fundamental challenge on large scale data analytics. Currently, the most common large-scale computational infrastructures in science are shared High-Performance Computing (HPC) systems. Such systems are usually designed primarily to support computationally intensive batch jobs-e. g. for the simulation of physical processes-and are managed by specialized system administrators. This model leads to rigid constraints on the way these resources can be used. For instance, the installation of software must undergo approval and may be restricted, which contrasts with the needs in the analysis where a multitude of software components of various versions-and their dependencies-are needed, and where these need to be continuously updated.
, 2016). In addition, configuring the necessary software tools and chaining them together into a complete re-runnable analysis workflow commonly requires substantial ITexpertise, while creating portable and fault-tolerant workflows with a robust audit trail is even more difficult. Metabolomics has already benefited from cloud-based systems enabling the users certain preprocessing and main downstream analysis on e. g. MS data. Examples of such systems are XCMS ONLINE  (Warth et al. , 2017) , MetaboAnalyst  (Xia et al. , 2012) , Chorus (chorusproject. org) and The Metabolomics Workbench  (Sud et al. , 2016)  (www. metabolo micsworkbench. org) which provide tools that scale with computational demands. In this manuscript, we present a method that uses components for data analysis encapsulated as microservices and connected into computational workflows to provide complete, ready-to-run, reproducible data analysis solutions that can be easily deployed on desktop computers as well as public and private clouds.
Our work contrasts to previously reported research environments, sometimes termed Virtual Research Environments  (Allan, 2009; Candela et al. , 2013) , Scientific Gateways  (Lawrence et al. , 2015)  and Virtual Labs  (Waldrop, 2013) , in that it encompasses the complete setup of the computational infrastructure and frameworks to run analysis in a wide range of environments; however our approach requires virtually no involvement in the setup and no special IT skills from the user. The methodology provides a framework for rapid and efficient integration of new tools and developing scalable, and interoperable workflows, supporting multiple workflow engines such as Galaxy  (Goecks et al. , 2010)  and Luigi (https://github. com/spotify/luigi). We validate the method on four metabolomics studies and show that it enables scalable and interoperable data analysis.
Microservices A detailed description of the methods is present in Supplementary Method S1. Briefly, in order to construct a microservice architecture for metabolomics we used Docker  (Merkel, 2014)  (https://www. docker. com/) containers to encapsulate software tools. Tools are developed as open source and are available in a public repository such as GitHub (https://github. com/), and the PhenoMeNal project containers are built and tested on a Jenkins continuous integration (CI) server (http://phenomenal-h2020. eu/jenkins/). Containers are assembled in different branches using the git versioning system. Builds originating from the development branch of each container repository give rise to container images tagged as 'development'; builds coming from the master branches result in release images.
Virtual Research Environment (VRE) We developed a Virtual Research Environment (VRE) which uses Kubernetes (https://kubernetes. io/) for orchestration of the containers, including initialization and scaling of jobs based on containers, abstractions to file system access for running containers, exposure of services, as well as rescheduling of failed jobs and long running services. Kubernetes was chosen over other frameworks such as Docker Swarm because of its larger momentum and that it is more widely used in production environments. Docker also provides Kubernetes as part of their Enterprise solutions (and even now the community ones). To enable convenient instantiation of a complete virtual infrastructure, we developed KubeNow (https://github. com/kube now/KubeNow)  (Capuccini et al. , 2018)  which includes instantiation of compute nodes, shared file system storage, networks, configure DNS, operating system, container implementation and orchestration tools, including Kubernetes, on a local computer or server.
Demonstrators We validated our method in the field of metabolomics using four demonstrators. Demonstrators 1 and 2 showcase scalability and interoperability of our microservice-based architecture whereas Demonstrators 3 and 4 exemplify flexibility to account for new application domains, showing the architecture is domain-agnostic. Demonstrator 1: Scalability of microservices in a cloud environment. The objective of this analysis was to demonstrate the computational scalability of an existing workflow on a large dataset [Metabolomics data have been deposited to the EMBL-EBI MetaboLights database  (Haug et al. , 2013)  with the identifier MTBLS233  (Ranninger et al. , 2016). The complete dataset can be accessed here https://www. ebi. ac. uk/metabolights/MTBLS233]. The experiment includes 528 mass spectrometry samples from whole cell lysates of human renal proximal tubule cells that were pre-processed through a five-step workflow (consisting of peak picking, feature finding, linking, file filtering and exporting) using the OpenMS software  (Sturm et al.
, 2008). This preprocessing workflow was reimplemented using Docker containers and run using the Luigi workflow engine. Scalability of concurrent running tools (on 40 Luigi workers, each worker receives tasks from the scheduler and executes them) was measured using weak scaling efficiency (WSE), where the workload assigned to each worker stays constant and additional workers are used to solve a larger total problem. Demonstrator 2: Interoperability of microservices. The objective of this analysis was to demonstrate interoperability as well as to present a real-world scenario in which patients' data are processed using a microservices-based platform. We used a dataset consisting of 37 clinical cerebrospinal fluid (CSF) samples including thirteen relapsing-remitting multiple sclerosis (RRMS) patients and 14 secondary progressive multiple sclerosis (SPMS) patients as well as 10 non-multiple sclerosis controls.
26 quality controls (19 blank and 7 dilution series samples) were also added to the experiment. In addition, 8 pooled CSF samples containing MS/MS data were included in the experiment for improving identification [Metabolomics data have been deposited to the EMBL-EBI MetaboLights database with the identifier MTBLS558. The complete dataset can be accessed here https://www. ebi. ac. uk/metabolights/MTBLS558]. The samples were processed and analyzed on the Galaxy platform  (Goecks et al. , 2010) , running in a VRE behind the Uppsala University Hospital firewall to be compliant with local ELSI (Ethics, Legal, Social implications) regulations. Demonstrator 3: 1 D NMR-analysis workflow. The purpose of this demonstrator was to highlight the fact that the microservice architecture is indeed domain-agnostic and is not limited to a particular assay technology.
This NMR-based metabolomics study was originally performed by  Salek et al. (2007)  on urine of type 2 diabetes mellitus (T2DM) patients and controls [Metabolomics data have been deposited to the EMBL-EBI MetaboLights database with the identifier MTBLS1. The complete dataset can be accessed here https://www. ebi. ac. uk/metabolights/MTBLS1]. In total, 132 samples (48 T2DM and 84 controls) were processed using a Galaxy workflow performing conversion, preprocessing, multivariate data analysis and result visualization. Demonstrator 4: Start-to-end fluxomics workflow. The purpose of this demonstrator was to show the integrated use of separately developed tools covering subsequent steps of the study of metabolic fluxes based on 13 C stable isotope-resolved metabolomics (SIRM)  (Buescher et al. , 2015; King et al.
, 2015; Niedenfu ¨hr et al. , 2015). Here we implemented the analysis of flux distributions in HUVEC cells under hypoxia [Metabolomics data have been deposited to the EMBL-EBI MetaboLights database with the identifier MTBLS412. The complete dataset can be accessed here https://www. ebi. ac. uk/metabolights/MTBLS412], from raw mass spectra contained in netCDF files, using a workflow implemented in Galaxy including reading and extraction of the data, correcting the evaluated mass spectra for natural isotopes and computing steady-state distribution of 13 C label as function of steady-state flux distribution.
Availability and implementation The PhenoMeNal consortium maintains a web portal (https://portal. phenomenal-h2020. eu) providing a GUI for launching VREs using KubeNow  (Capuccini et al. , 2018)  on a selection of the largest public cloud providers, including Amazon Web Services, Microsoft Azure and Google Cloud Platform, or on private OpenStack-based installations. The Wiki containing documentation is also hosted on GitHub https://github. com/phnmnl/phenomenal-h2020/wiki. The PhenoMeNal Portal can be reached at https://portal. phenomenal-h2020. eu. The public instance of Galaxy is accessible at https://pub lic. phenomenal-h2020. eu. The containers provisioned by PhenoMeNal comprise tools built as open source software that are available in a public repository such as GitHub, and are subject to continuous integration testing.
The containers that satisfy testing criteria are pushed to a public container repository, and containers that are included in stable VRE releases are also pushed to Biocontainers (da Veiga  Leprevost et al. , 2017). The GitHub repository https://github. com/phnmnl/hosts the source code of all development projects. Source code and documentation are available under the terms of the Apache 2. 0 license. Integrated open source projects are available under the respective licensing terms. The Demonstrators can be obtained from: Demonstrator 1: https:// github. com/phnmnl/MTBLS233-Jupyter; Demonstrator 2: https:// public. phenomenal-h2020. eu/u/phenoadmin/w/metabolomics-lcmsmsprocessing-quantification-annotation-identification-and-statistics-1; Demonstrator 3: https://public. phenomenal-h2020. eu/u/phenoad min/w/metabolomics-nmr-rnmr1d-metabolights-data-processing-andplot; Demonstrator 4: https://public.
Results We developed a VRE based on a microservices architecture encapsulating a large suite of software tools for performing metabolomics data analysis (see Supplementary Table  S1 ). Scientists can interact with the microservices programmatically via an Application Programming Interface (API) or via a web-based graphical user interface (GUI), as illustrated in Figure  1. To connect microservices into computational workflows, the two frameworks Galaxy  (Goecks et al. , 2010)  and Luigi (https://github. com/spotify/luigi) were adapted to execute jobs on Kubernetes. Galaxy is a web-based interface for individual tools and allows users to share workflows, analysis histories and result datasets. Luigi on the other hand focuses on scheduled execution, monitoring, visualization and the implicit dependency resolution of tasks  (Leipzig, 2017).
These basic infrastructure services, together with the Jupyter notebook  (Kluyver et al. , 2016)  interactive programming environment, are deployed as long-running services in the VRE, whereas the other analysis tools are deployed as transient compute jobs to be used on-demand. System and client applications were developed for launching the VRE on desktop computers, public and private cloud providers, automating all steps required to instantiate the virtual infrastructures. Demonstrator 1: Scalability of microservices in a cloud environment. The Diagram of scalability-testing on the metabolomics dataset is illustrated in Figure  2. The analysis resulted to WSE of 88% with an execution time of approximately four hours (online methods, Supplementary Fig. S2 ), compared with the ideal case of 100% where linear scaling is achieved if the run time stays constant while the workload is increased.
In addition, the final result of the workflow (online methods, Supplementary Fig. S3 ) was identical to that presented by the original MTBLS233 study  (Ranninger et al. , 2016)  in negative ionization mode. However, in the positive ionization mode, one m/z feature was found in a different group (m/z range 400-1000) than it was originally reported by. Demonstrator 2: Interoperability of microservices. We developed a start to end workflow for pre-processing and statistical analysis of LC-MS metabolomics data (Fig. 3 ). The workflow allows seamless integration of six major metabolomics data analysis components (26 steps) each was already implemented in independent software suites: noise reduction and filtering [OpenMS  (Rost et al. , 2016) ], quantification, alignment and matching [XCMS Complex applications are divided into smaller, focused and well-defined (micro-) services.
These services are independently deployable and can communicate with each other, which allows to couple them into complex task pipelines, i. e. data processing workflows. The user can interact with the framework programmatically via an Application Program Interface (API) or via a graphical user interface (GUI) to construct or run workflows of different services, which are executed independently. Multiple instances of services can be launched to execute tasks in parallel, which effectively can be used to scale analysis over multiple compute nodes. When run in an elastic cloud environment, virtual resources can be added or removed depending on the computational requirements  (Smith et al. , 2006) ], filtering of biological non-relevant signals (R), annotation of signals [CAMERA  (Kuhl et al.
, 2012) ], identification [MetFrag  (Wolf et al. , 2010) ], statistics [Workflow4Metabolomics  (Giacomoni et al. , 2015) ]. The result of the workflow (multivariate analysis) showed a clear difference in the metabolic constitution between the three disease groups of RRMS, SPMS and non-multiple sclerosis controls (Fig. 4A ). In addition, the univariate analysis resulted in a total of three metabolites being significantly altered (p < 0. 05) between multiple sclerosis subtypes and control samples, namely alanyltryptophan and indoleacetic acid with higher and linoleoyl ethanolamide with lower abundance in both RRMS and SPMS compared to controls (Fig. 4B ). Demonstrators 3 and 4: Domain agnosticity (NMR and fluxomics workflows). We developed a workflow for analysis of 1 D NMR data. The workflow consisted of automatic downloading NMR vendor data (and metadata) from MetaboLights database followed by format standardization, spectral processing and statistical analysis.
We processed a NMR dataset (demonstrator 3) resulting to quantification of a total of 726 features which were used to perform Orthogonal Projections to Latent Structures Discriminant Analysis (OPLS-DA). This resulted in a clear separation between T2DM and controls (Fig. 5 ), similar to that of previous findings  (Salek et al. , 2007). Lastly, we designed a workflow for analyzing metabolite metabolic fluxes. The workflow integrated four main steps including data extraction, data correction, calculation of flux distribution and visualization. Using this workflow (Fig. 6 ), we achieved detailed description of the magnitudes of the fluxes through the reactions accounting for glycolysis and pentose phosphate pathway.
At first sight microservices might seem similar to above mentioned SOA web services, but microservices can with great benefit be executed in virtual environments (abstracting over OS and hardware architectures) in such a way that they are only instantiated and executed on-demand, and then terminated when they are no longer needed. This makes such virtual environments inherently portable and they can be launched on demand on different platforms A) The preprocessing workflow is composed of 5 OpenMS tasks that were run in parallel over the 12 groups in the dataset using the Luigi workflow system. The first two tasks, peak picking (528 tasks) and feature finding (528 tasks), are trivially parallelizable, hence they were run concurrently for each sample. The subsequent feature linking task needs to process all of the samples in a group at the same time, therefore 12 of these tasks were run in parallel.
In order to maximize the parallelism, each feature linker container (microservice) was run on 2 CPUs. Feature linking produces a single file for each group, that can be processed independently by the last two tasks: file filter (12 tasks) and text exporter (12 tasks), resulting in total of 1092 tasks. The downstream analysis consisted of 6 tasks that were carried out in a Jupyter Notebook. Briefly, the output of preprocessing steps was imported into R and the unstable signals were filtered out. The missing values were imputed and the resulting number of features were plotted. B) The weak scaling efficiency plot for Demonstrator 1. Given the full MTBLS233 dataset, the preprocessing was run on 40 Luigi workers. Then for 1/4, 2/4, 3/4 of MTBLS233, the analysis was run again on 10, 20 and 30 workers respectively.
For each run, we measured the processing time T10, T20, T30 and T40, and we computed the WSEn ¼ T10/Tn for n ¼ 10, 20, 30, 40. The WSE plot shows scalability up to 40 CPUs, where we achieved 88% scaling efficiency. The running time for the full dataset (a total of 1092 tasks) on 40 workers was 4 hours Fig. 3. Overview of the workflow used to process multiple-sclerosis samples in Demonstrator 2, where a workflow was composed of the microservices using the Galaxy system. The data was centroided and limited to a specific mass over charge (m/z) range using OpenMS tools. The mass traces quantification and retention time correction was done via XCMS  (Smith et al. , 2006). Unstable signals were filtered out based on the blank and dilution series samples using an in-house function (implemented in R).
Annotation of the peaks was performed using CAMERA  (Kuhl et al. , 2012). To perform the metabolite identification, the tandem spectra from the MS/MS samples in mzML format were extracted using MSnbase and passed to MetFrag. The MetFrag scores were converted to q-values using Passatutto software. The result of identification and quantification were used in 'Multivariate' and 'Univariate' containers from Workflow4Metabolomics  (Giacomoni et al. , 2015)  to perform Partial Least Squares Discriminant Analysis (PLS-DA) (e. g. a laptop, a powerful physical server or an elastic cloud environment). A key aspect is that workflows of microservices are still executed identically, agnostic of the underlying hardware platform. Container-based microservices provide a wide flexibility in terms of versioning, allowing the execution of newer and older versions of each container as needed for reproducibility.
Since all software dependencies are encompassed within the container, which is versioned, the risk of workflow failure due to API changes is minimized. An orchestration framework such as Kubernetes further allows for managing errors in execution and transparently handles the restarting of services. Hence, technology has caught up with service-oriented science, and microservices have taken the methodology to the next level, alleviating many of the previous problems related to scalability, portability and interoperability of software tools. This is advantageous in the context of omics analysis, which produces multidimensional datasets reaching beyond gigabytes, on into terabytes, leading to ever-increasing demand on processing performance  (Marx, 2013; Schadt et al. , 2010). However, containerization does not address how services communicate with each other, but this has to be implemented inside the container itself.
Traditional web services addressed this by standardizing the messaging protocol and public-facing interfaces (e. g. SOAP and WSDL)  (Stockinger et al. , 2008) , while in a containerized environment Representational State Transfer (REST)  (Fielding, 2000)  or passing files by reference to a shared file system is more common. In Demonstrator 1, we showed that microservices enable highly efficient and scalable data analyses by executing individual modules in parallel, and that they effectively harmonize with on-demand elasticity of the cloud computing paradigm. The reached scaling efficiency of 88% indicates remarkable performance achieved on generic cloud providers. Furthermore, although our results in positive ionization model was slightly different to that of  Ranninger et al. (2016) , the results of our analysis were replicable regardless of the platform used to perform the computations.
In addition to the fundamental demand for high performance, the increased throughput and complexity of omics experiments has A) The PLS-DA results suggest that the metabolite distribution in the RRMS and SPMS samples are different to controls. B) Three metabolites were identified as differentially regulated between multiple sclerosis subtypes and control samples, namely Alanyltryptophan and Indoleacetic acid with higher and Linoleoyl ethanolamide with lower abundance in both RRMS and SPMS compared to controls. Abbr. , RRMS: relapsing-remitting multiple sclerosis, SPMS: secondary progressive multiple sclerosis Fig. 5. Overview of the NMR workflow in Demonstrator 3. The raw NMR data and experimental metadata (ISATab) was automatically imported from the Metabolights database and converted to open source nmrML format.
The preprocessing was performed using the rnmr1d package part of nmrprocflow tools. All study factors were imported from MetaboLights and were fed to the multivariate node to perform an OPLS-DA led to a large number of sophisticated computational tools  (Berger et al. , 2013) , which in turn necessitates integrative workflow engines  (Atkinson et al. , 2017; Di Tommaso et al. , 2017; Liew et al. , 2016). In order to integrate new tools in such workflow engines, compatibility of the target environment, tools and APIs needs to be considered  (Di Tommaso et al. , 2017). Containerization facilitates this by providing a platform-independent virtual environment for developing and running the individual tools. However, the problem of compatibility between tools/APIs and data formats remains and needs to be tackled by international consortia  (Wilkinson et al.