dataesr/scientific-paragraphs-categorization huggingface dataset:
Problem Definition This section introduces some important terms, i. e. SWf, SWf fragment and multisite cloud and defines the scheduling problem in the multisite cloud. A SWf is described as a Directed Acyclic Graph (DAG) denoted by W(V,E). Let V = {v 1 , v 2 ,. , v n } be a set of vertices, which are associated with the scientific data processing activities and E = {e i, j : v i , v j ∈ V and v j consumes the output data of v i } be a set of edges that correspond to dependencies between activities in V. Activity v j is the following activity of Activity v i and Activity v i is a preceding activity of Activity v j. The dependencies can be data or control dependencies. Compared to data dependencies, fewer data are transferred in control dependencies. The transferred data in a control dependency is the configuration parameters for activity execution while the transferred data in a data dependency is the input data to be processed by the following activity.
A fragment dependency f e i, j represents that fragment w f j processes the output data of fragment w f i. f e i, j is the input dependency of w f j and output dependency of w f i. A fragment can be denoted by w f (V, E, D). V represents the activities, E represents the dependencies and D represents the input data of the workflow fragment. We denote the SWf execution environment by a configured multisite cloud 1 MS (S ), which consists of a set of sites S. A multisite cloud configuration defines the instances of VMs and storage resources for cloud users in a multisite cloud. One site s i ∈ S is composed of a set of Web domains. A Web domain contains a set of VMs, shared storage resources, and stored data. In this paper, we assume that one site contains only one Web domain for the execution of a SWf.
1. In the paper, we use SSVP (see Section 5. 2) to generate VM provisioning plans. The stored data constraints can be represented as a matrix (as the one presented below), and its cell values are known before execution, where each row a i represents an activity, each column s j represents a cloud site, and (a i , s j ) = 1 means that a i needs to read the data stored at s j. s 1 s 2 s 3 a 1 1 0 0 a 2 0 0 1 a 3 0 1 0.
The deployment of a VM is to create a VM under a user account in the cloud. The deployment of the VM defines the type and location, namely the cloud site, of the VM. The initialization of a VM is the process of starting the VM, installing programs and configuring parameters of the VM, so that the VM can be used for executing the tasks of fragments. The multisite data transfer module transfers the input data of fragments to the site. Finally, the single site execution module starts the execution of the fragments at each site. This can be realized by an existing single site SWfMS, e. g. Chiron  [26]. Within a single site, when the execution of its fragment is finished and the output data is moved to other sites, the VMs are shut down. When the execution of the fragment is waiting for the output data produced by other sites and the output data at this site are transferred to other corresponding sites, the VMs are also shut down to avoid the useless monetary cost.
When the necessary data is ready, the VMs are restarted to continue the execution of the fragment. In a multisite cloud, there are two types of sites, i. e. coordinator and participant. The coordinator is responsible for coordinating the execution of fragments at different participants. Two modules, namely workflow partitioner and multisite scheduler, are implemented at the coordinator site. Both the coordinator and participants execute the scheduled fragments. The initialization module and single site execution module are implemented at both the coordinator and participants.
1. 5) In this formula, CP(S W f ) represents the critical path of Workflow S W f , which can be generated by the method proposed by Chang et al. [11]. A critical path is a path composed of a set of activities with the longest average execution time from the start activity to the end activity  [11]. In a SWf, the start activity is the activity that has no input dependency and the end activity is the activity that has no output dependency. Similarly, CP(w f ) represents the critical path of Fragment w f. The workload workload(a i , inputData) of an activity a i with a specific amount of data inputData is estimated by users according to the features of the SWf. DesiredT ime is the desired execution time for the whole workflow, defined by user. Since the time to execute a fragment or a SWf is similar to that of the executing the activities in the critical path, we calculate the desired time for a fragment as the part of the time to execute the same workload of activities in the critical path of the SWf as that of the fragment.
In this formula, the time to wait for the input data produced by the activities executed at another site (Site s o ) is not considered since this time is considered in that of the fragment executed at Site s o. The multisite SWfMS needs to provision m (determined by SSVP) VMs to execute Fragment w f at a single site. As explained in Section 4, to provision a VM is to deploy and to initialize a VM at a cloud site. The time to provision the VMs at a single site is estimated by Formula 5. 1. 7. InitializationT ime(w f, s) = m * InitializationT ime (5. 1. 7) InitializationT ime represents the average time to provision a VM. The value of InitializationT ime can be configured by users according to the cloud environment, which can be obtained by measuring the average time to start, install the required programs and configure 2 -3 VMs.
Except Activities 6. 1 -6. 6, all the other activities are data-intensive. However, the time to process data is small while the time to transfer and store the input or output data takes much time. In Figure  2 , "read data" represents that one activity just reads the stored data without modifying it and that the activity should be executed at the corresponding site to read data since the stored data is too big to be moved and can be accessed only within the corresponding site because of configurations, e. g. security configuration of a database. The stored data constraints are defined by the following matrix (the activities not listed in the matrix are not fixed). s 1 s 2 s 3 a 6. 1 1 0 0 a 6. 2 1 0 0 a 6. 3 0 1 0 a 6. 4 0 1 0 a 6. 5 0 0 1 a 6. 6 0 0 1.
Experimental Evaluation In this section, we present an experimental evaluation of the VM provisioning plan generation (SSVP) and fragment scheduling (ActGreedy) algorithms. All experiments are based on the execution of the SciEvol SWf in Microsoft Azure multisite cloud. First, we compare SSVP with GraspCC. Then we compare ActGreedy with LocBased and SGreedy, as well as with two general algorithms, i. e. Genetic and Brute-force. In the experiments, we consider three Azure  [2]  sites to execute SciEvol SWf, namely West Europe as Site 1, Japan West as Site 2, Japan East as Site 3. During the experiments, the the life circle of VM is composed of creation, start, configuration, stop and deletion. The creation, start, stop and deletion of a VM is managed by using Azure CLI  [3]. The configuration of VM is realized by Linux SSH command.
Each tier of VM  In the experiments, we use 100, 500, 1000 fasta files generated from the data stored in a genome database  [7]   [8]. The programs used are: mafft (version 7. 221) for Activity 1, Read-Seq 2. 1. 26 for Activity 2, raxmhpc (7. 2. 8 alpha) for Activity 4, pamlX1. 3. 1 for Activities 6. 1 -6. 6, in-house script for Activity 3 and Activity 8, and Activity 5 and Activity 7 exploit a PostgreSQL database management system to process data. The percentage of the workload, i. e. α in Formula 5. 1. 10, that can be parallelized is 96. 43%. In addition, the input data of the SWf is stored at a data server of Site 3, which is accessible to all the sites in the cloud using SCP command (a Linux command). The estimated workload (in GFLOP) of each activity of SciEvol SWf for different numbers of input fasta files is shown in Table  3  and the estimated amount of data transferred in each dependency of SciEvol SWf is listed in Table  4.
Acknowledgment Work partially funded by  EU H2020 Programme  and  MCTI/RNP-Brazil  ( HPC4E  grant agreement number  689772 ),  CNPq ,  FAPERJ , and  INRIA  ( MUSIC  project),  Microsoft  ( ZcloudFlow  project) and performed in the context of the  Computational Biology Institute  (www. ibc-montpellier. fr). We would like to thank  Kary Ocaña  for her help in modeling and executing the SciEvol SWf.
Background: Scaffolding is an essential step in the genome assembly process. Current methods based on large fragment paired-end reads or long reads allow an increase in contiguity but often lack consistency in repetitive regions, resulting in fragmented assemblies. Here, we describe a novel tool to link assemblies to a genome map to aid complex genome reconstruction by detecting assembly errors and allowing scaffold ordering and anchoring. Results: We present MaGuS (map-guided scaffolding), a modular tool that uses a draft genome assembly, a Whole Genome Profiling™ (WGP) map, and high-throughput paired-end sequencing data to estimate the quality and to enhance the contiguity of an assembly. We generated several assemblies of the Arabidopsis genome using different scaffolding programs and applied MaGuS to select the best assembly using quality metrics.
Then, we used MaGuS to perform map-guided scaffolding to increase contiguity by creating new scaffold links in low-covered and highly repetitive regions where other commonly used scaffolding methods lack consistency. Conclusions: MaGuS is a powerful reference-free evaluator of assembly quality and a WGP map-guided scaffolder that is freely available at https://github. com/institut-de-genomique/MaGuS. Its use can be extended to other highthroughput sequencing data (e. g. , long-read data) and also to other map data (e. g. , genetic maps) to improve the quality and the contiguity of large and complex genome assemblies.
The ability of these methods to produce pseudo-molecules also was tested, as reported in Popseq  [20]  and Allmaps  [21]. Allmaps infers the sizes of gaps using the relation between the local recombination rate and the physical distance between two adjacent genetic markers; however, the estimations can be inconsistent considering the inaccuracy of the recombination rate. Hybrid strategies, combining WGS and genome map data, are likely to help increase the quality of the assembled genome sequence. With this in mind, we developed MaGuS, a modular program that combines a genome map and WGS data. MaGuS can anchor a draft assembly onto a genome map for two applications: quality assessment of a draft assembly by calculating novel metrics, and improvement of the contiguity of a draft assembly based on evidence provided by a genome map and highthroughput screening (HTS) data.
We set the k-mer size to 31 bp which is more stringent than the bowtie and bwa mem default parameter, k = 28 and k = 19 respectively. We set the minimum number of link to five which corresponded to the default parameter of BESST and SSPACE. For SOAPdenovo2, we ran the map command with parameter '-k 31' , the scaf command with parameter '-L 500' , and set the minimum number of links in the configuration file as 'pair_num_cutoff = 5'. For SSPACE, we manually set the bowtie k-mer size '-l 31' and ran the program with parameter '-k 5'. For SGA and BESST, we first aligned the MP reads onto the contigs using BWA aln  [29]  with parameter '-l 31'. For SGA, the links file was created using the sga-bam2de command with parameters '-n 5 -m 500 -mina 31 -k 31'. The astat file was generated setting '-m 500'.
The scaf file and the corresponding FASTA file were both created with parameters '-m 500'. For BESST, we chose the optimal k-mer size used for the contig assembly as '-K 91' and ran the program with parameter '-e 5'. For each program, we selected the scaffolds that were over 2 kb in length. For OPERA-LG, we set the k-mer size for scaffolding with option 'kmer = 91'. The minimum contig size required for the scaffolding step was fixed as 500 bp with the parameter 'contig_size_threshold = 500'. Finally, the number of links to validate a connection between two contigs was assigned with the parameter 'cluster_threshold = 5'. To evaluate the quality of each assembly, we used QUAST v2. 3, a popular program based on Nucmer. In the presence of a trusted reference, QUAST aligns with Nucmer the assembly to the provided reference and generates quality metrics.
We observed several inconsistancies in the QUAST output. After discussions with the QUAST authors, the source code of QUAST was modified to avoid, as much as possible, the detection of misassemblies (relocation, translocation, and inversion) that correspond to false positives. Because Nucmer generated spurious alignments lower than 5 kb in highly repetitive regions, the minimum alignment length in both parts of a misassembly was set to 5 kb. Moreover, the gap or overlap size threshold length was increased to 5 kb to detect relocations. By default, QUAST reports misassemblies found within a scaffold only if at least 50 % of the scaffold is aligned. We modified this parameter to report all misassemblies regardless of the aligned fraction of a scaffold.
Implementation of MaGuS MaGuS was implemented in a Perl program based on five modules: wgp2map, which performs the anchoring and creates a MaGuS-format map that contains the anchoring information; map2qc, which evaluates the quality of the assembly; map2link, which creates the map-links between scaffolds; pairs2links, which validates the map-links, orients the scaffolds, estimates the gap size, and creates a link. de file; and links2scaf, which runs the SGA scaffolding programs and creates the final assembly.
Based on the QUAST NA50 and NA75 metrics, we ranked the assemblies from the highest to lowest quality as BESST, OPERA-LG, SSPACE, SOAP-denovo2, and SGA. We used the WGP map to provide a reference-free approach that evaluates the quality of the five assemblies. We applied the wgp2map and map2qc modules of MaGuS to calculate the length of all collinear segments (Fig. 1b ) and generated Anx values (Table  1 , Fig. 2a ). Considering the MaGuS An50 and the An75 metrics, the ranking of the assemblies was the same as the ranking using the QUAST NA50 and NA75 metrics. The NAx and Anx values were strongly correlated (R 2 > 0. 96) for the five assemblies (Fig. 2c ), which allowed us to consider using the MaGuS Anx metrics to compare assembly quality. Selecting the appropriate bioinformatics tools to perform genome de novo assembly is difficult and often depends on the genome complexity and on the sequencing technology used.
The simulation of artificial WGP maps containing intra and inter-BAC contig errors at different rates showed that the map-guided scaffolding accuracy is not affected by errors whereas the contiguity depends closely on the noise of the WGP map (for intra as well as inter-BAC contig error rate over 0. 1). As the WGP data of Arabidopsis have an intra and inter-BAC contig error rate under 0. 1, we can consider that the Arabidopsis WGP dataset was clean enough to be efficiently used by MaGuS for a guided scaffolding.
Conclusions The method presented here and implemented in MaGuS enabled the evaluation of the quality and the scaffolding of a draft genome assembly using a physical map and HTS data. Its application to Arabidopsis with a WGP map provides a first example of its efficiency in reconstructing a eukaryotic genome. Evaluating the quality of a genome assembly is necessary in order to increase the accuracy of downstream analyses, such as genome annotation or comparative genomic analyses. De novo assembly projects often lack a genome reference and different ways to assess the assembly quality have been investigated  [2, 34]  using either the HTS data used for the assembly or a genome map. The latter remains a very good independent source of information for this task. From this perspective, we developed the map2qc module of MaGuS to provide assembly quality metrics.
Motivation: With the continued improvement of requisite mass spectrometers and UHPLC systems, Hydrogen/Deuterium eXchange Mass Spectrometry (HDX-MS) workflows are rapidly evolving towards the investigation of more challenging biological systems, including large protein complexes and membrane proteins. The analysis of such extensive systems results in very large HDX-MS datasets for which specific analysis tools are required to speed up data validation and interpretation. Results: We introduce a web application and a new R-package named 'MEMHDX' to help users analyze, validate and visualize large HDX-MS datasets. MEMHDX is composed of two elements. A statistical tool aids in the validation of the results by applying a mixed-effects model for each peptide, in each experimental condition, and at each time point, taking into account the time dependency of the HDX reaction and number of independent replicates.
, 2015; Marciano et al. , 2014; Wei et al. , 2014). The use of improved HDX-MS workflows enables the structural analysis of larger protein systems such as antigen-antibody complexes (> 180 kDa) or integral membrane proteins, in a more routine way  (Bertoldi et al. , 2016; Chung et al. , 2011; Faleri et al. , 2014; Malito et al. , 2014; Malito et al. , 2013). The characterization of such systems results in very complex HDX-MS datasets, for which specific non-commercial analytical software (e. g. HXExpress, HeXicon, ExMS, HDXFinder, etc. ), as well as commercial platforms (e. g. DynamX and HDExaminer), have been developed  (Guttman et al. , 2013; Hamuro et al. , 2003; Kan et al. , 2011; Kreshuk et al. , 2011; Lindner et al. , 2014; Lou et al. , 2010; Miller et al. , 2012; Weis et al. , 2006).
Such software extract deuterium incorporation information from (un)processed raw m/z data files and produce deuterium uptake curves that can pinpoint regions of interest on protein 3D structures. However, many of these tools do not integrate statistical approaches and use the absolute difference of deuterium uptake to evaluate the significance between conditions. Hydra (or Mass Spec Studio) was the first standalone application to introduce Student's t-test and P-values to statistically evaluate the difference across HDX-MS experiments  (Rey et al. , 2014; Slysz et al. , 2009). Alternatively,  Houde et al. (2011)  calculated confidence limits based on the experimental uncertainty of measuring deuterium uptake across replicates. In this context, two distinct confidence limits were calculated manually using either the differences of deuterium uptake or the summed value of HDX differences measured for each peptide, in each condition, and at each time point.
Lastly, HDX Workbench includes a two-tailed t-test and Tukey multiple comparison procedure for the statistical cross-comparison of two or more datasets  (Pascal et al. , 2009; Pascal et al. , 2007 Pascal et al. , , 2012)). Despite significant efforts to design requisite statistical tools, the aforementioned software solutions are suitable to analyze HDX-MS data at one time point only, failing to account for the time dependency of the HDX-MS reaction. Thus,  Liu et al. (2011)  proposed a multiple regression or ANCOVA model. By deriving a statistical test based on the model parameters, they evaluated the significant difference between two groups under comparison, for all peptides in the dataset, and across all independent replicates. Expanding on this, we introduce a web application named 'MEMHDX' (Mixed-Effects Model for HDX experiments) to aid in the rapid statistical validation and visualization of large HDX-MS datasets.
MEMHDX uses a linear mixed-effects model where replicates are considered as random effects. This accounts for both the time dependency and the variability across replicates. Moreover, instead of testing the variation in global deuterium exchange between two experimental conditions, we propose to calculate two individual P-values for each peptide. First, the difference between conditions is measured (P-value for the 'Magnitude of Delta Deuterium (DD)'), followed by the evolution of the deuterium uptake behavior over the time course of the experiment (P-value for the 'Change in dynamics'). MEMHDX, therefore, allows the clustering of each peptide in the dataset based on these two respective P-values. A user-friendly interface developed with Shiny by RStudio facilitates the use of the application, allowing the user to easily visualize and compare the relative deuterium incorporation across the entire protein sequence and 3D structure.
The statistical inference The 'nmle' R package was utilized to estimate the vectors of fixed effects (b) and u r (http://CRAN. R-project. org/package¼nlme). The restricted maximum-likelihood method was used to fit the linear mixed-effects model. To evaluate the statistical significance of the condition and the interaction on the rate of deuterium incorporation of the protein, two P-values were calculated per peptide, using two individual Wald tests. The condition-associated P-value (hereafter referred to as P-value Magnitude_of_Delta_D ) tests the null hypothesis of there being no difference in deuterium uptake between conditions 1 and 2; the interaction-associated P-value (hereafter referred to as P-value Change_in_dynamics ) tests the null hypothesis of there being no change in the deuterium uptake behavior between conditions 1 and 2 and takes into account the time dependency of the HDX reaction (see Comment 1 of Supplementary material).
Results and discussion MEMHDX was developed to complement the HDX-MS pipeline commercialized by Waters Corporation. The software is fully compatible with the main output generated by DynamX (i. e. cluster data), but has been designed to handle data from any HDX-MS platform, as long as the input file is structured in. csv format and with the appropriate architecture, i. e. , columns listed in Panel 1 of Supplementary Figure  S1A. MEMHDX was evaluated using our recently published differential HDX-MS dataset generated with the C-terminal Repeat-in-toxin Domain (RD, 701 residues) of the CyaA toxin  (O'Brien et al. , 2015). Briefly, we used HDX to identify and locate secondary structural elements in the intrinsically disordered Apo-RD protein, and followed its transition to a more compact and folded state upon calcium binding.
Pre-processing and dataset quality control A total of 602 peptic peptides of RD were identified by ProteinLynX Global Server (Waters Corporation) using the default search parameters, of which 198 remained after filtering by DynamX. Following this, 162 were selected for HDX analysis and covered 98. 4% of the RD sequence (Supplementary Fig. S2 ). Considering 162 peptides, 1 charge state, 3 replicates, 14 time points (including the unlabeled control) and 2 conditions, we have up to 13 608 unique data points in the complete test dataset. This demonstrates the complexity of datasets acquired using modern HDX-MS pipelines, the processing and interpretation of which is extremely challenging and time consuming by traditional, manual means. The current version of MEMHDX can only handle one unique charge state per peptide.
A pre-processing step is required by the user to select the most appropriate charge state for analysis. The quality determination of the dataset is directly accomplished by MEMHDX. A Box and Whisker plot displaying the agreement across replicates is automatically generated (Supplementary Fig. S1B1 ). For each peptide, the calculated deuterium uptake values (Y a,i,r,t ) are averaged across all time points in each respective state. This comparison is possible as each replicate is considered as an independent variable. A scoring function (log-likelihood) is determined per peptide to easily control the quality of the fitting. MEMHDX includes a filtering option to sort each peptide according to this score and to remove those with a low fitting quality from the analysis.