dataesr/scientific-paragraphs-categorization huggingface dataset:
Sex of the larval offspring was determined later using sexlinked markers (identified using offspring with known sexes). DNA from the F0 grandmother (the F0 grandfather was lost), two F1 parents, and 69 of their F2 offspring was extracted using the DNeasy Blood and Tissue Kit (Qiagen). All samples were prepared as 300-bp insert size Illumina TruSeq libraries except for offspring 11, 16, 17, and 18, which were prepared as Nextera libraries due to low DNA quantities. Libraries were sequenced using 100-bp paired-end reads on an Illumina HiSeq2500 at the FAS Centre for Systems Biology genomics facility, Harvard University. Samples were sequenced over three HiSeq runs. Sequencing failed during sequencing of the second read for two libraries together containing 24 individuals; these libraries were resequenced, but the first run data were still used, with the second read truncated to 65 bases to include only bases of comparable quality to the first read.
Alignment and SNP calling Reads for parents and offspring were aligned to Hmel1. 1 using Stampy  (Lunter and Goodson 2011)  version 1. 0. 23 with options -substitutionrate = 0. 01 and -gatkcigarworkaround and converted from SAM to BAM format with the SortSam tool from Picard version 1. 117 (http:// broadinstitute. github. io/picard). Reads were aligned to the primary scaffolds (Hmel1-1_primaryScaffolds. fa), and haplotype scaffolds (Hmel_haplotype_scaffolds. fas) separately. Duplicate reads were removed using the Picard MarkDuplicates tool. Indels were realigned using the RealignerTargetCreator and IndelRealigner tools from GATK version 3. 2. 2 (DePristo et al. 2011). SNPs were called for each individual using the GATK HaplotypeCaller and combined into one final VCF file using GATK GenotypeGVCFs with options-annotateNDA and -max_alternate_alleles 30.
Merging genome HaploMerger version 20120810  (Huang et al. 2012 ) was used to collapse haplotypes in the H. melpomene genome. A scoring matrix for LASTZ (as used within HaploMerger) was generated using the lastz_D_Wrapper. pl script with-identity = 94. This scoring matrix was used for all runs of HaploMerger, including for the PacBio genome (see below). HaploMerger was run with default settings, except for setting-size = 20 in all_lastz. ctl, targetSize = 5,000,000 and querySize = 400,000,000 in hm. batchA. initiation_and_all_ lastz, and haploMergingMode="updated" in hm. batchF. refine_ haplomerger_connections_and_Ngap_fillings. Several scripts were written to make running HaploMerger easier. The new script runhm. pl executes one iteration of HaploMerger, running batch scripts A, B, C, E, F, and G, renaming output scaffolds with a given prefix, producing a final FASTA file concatenating merged scaffolds and unmerged scaffolds, and generating summary statistics (using summarizeAssembly.
py in PBSuite 14. 9. 9, http://sourceforge. net/projects/pb-jelly/;  English et al. 2012) , and an AGP file for the final FASTA (using bespoke script agp_from_fasta. py). The HaploMerger script hm. batchG. refine_unpaired_sequences was not used for the initial Hmel1. 1 and PacBio assembly merges, retaining all potentially redundant scaffolds in case they could be used for scaffolding later, but it was used to merge the haploid Hmel1. 1 assembly with the haploid PacBio assembly. The new script batchhm. pl runs runhm. pl iteratively until HaploMerger fails to merge any further scaffolds. It also runs a set of additional new scripts, map_merge. py, transfer_merge. py, and transfer_features. py, that document where the original genome parts are in the new genome. The map_merge. py script takes HaploMerger output and documents where the input genome scaffolds are in the merged output genome.
The transfer_merge. py script takes this transfer information and another transfer file, for example between the original version 1. 1 H. melpomene genome and the input genome, and computes the transfer from the original genome to the output genome. The transfer_features. py script then transfers linkage map markers, genes, and misassembly information to the new genome. HaploMerger sometimes merges scaffolds incorrectly, but has several mechanisms for users to manually edit its output. The hm. nodes file, which contains detected overlaps between scaffolds, can be manually annotated, with incorrect merges marked to be rejected. The revised hm. nodes file is then passed through the batchE script to update the merged scaffolds to ignore the incorrect merges. Incorrect merges in the Heliconius genome could be detected by comparing against the linkage map data.
A list of scaffolds that should not be merged was constructed over multiple merge attempts, and runhm. pl was used to edit the hm. nodes and run the batchE script automatically. HaploMerger merges scaffolds based on overlaps, and reports the parts contributing to merged scaffolds in the hm. new_scaffolds file, including which of the two overlapping parts has been included in the new genome. These choices sometimes broke genes, whereas choosing the other part would retain the annotated gene. runhm. pl can also take a GFF file as input, and check for broken genes in hm. nodes and hm. new_scaffolds, rejecting nodes if they break manually curated genes, and swapping parts in an overlap if it prevents gene breakage. It then runs the batchE and batchF to update the merged scaffolds. The Hmel1. 1 GFF files (heliconius_melpomene_v1.
Pacific Biosciences sequencing, error correction, and assembly A pupa from the H. melpomene genome strain from Gamboa, Panama was dissected, and DNA extracted using the QIAGEN HMW MagAttract kit. This pupa was taken after four generations of inbreeding, and came from the same generation as the F0 father used to construct the pedigree reported here, and the generation before the individuals used for the genome sequence itself. A Pacific Biosciences (PacBio) SMRTbell 25kb needle sheared library was constructed, sizeselected with 0. 375x SPRI beads, and sequenced using P4/C2 chemistry (180-min movie). PacBio subreads were self-corrected with PBcR [in Celera assembly v8. 3  (Berlin et al. 2015) ], run with options -length 200, -genomeSize 292,000,000), and separately corrected with the original genome strain Illumina (Sequence Read Archive accession SRX124669), 454 shotgun (SRX124544), and 454 3 kb mate-pair (SRX124545) sequencing data (using option -genomeSize 292,000,000).
Self-corrected and genomestrain-corrected reads were concatenated into one read set, and assembled with FALCON (https://github. com/PacificBiosciences/ falcon, commit bb63f20d500efa77f930c373105edb5fbe37d74b, April 2, 2015) with options input_type = preads, length_cutoff = 500, length_cutoff_pr = 500, pa_HPCdaligner_option="-v -dal4 -t16 -e. 70 -l1000 -s1000", ovlp_HPCdaligner_option="-v -dal32 -t32 -h60 -e. 95 -l500 -s1000, pa_DBsplit_option="-x500 -s50", ovlp_DBsplit_ option="-x500 -s50", falcon_sense_option="-output_multi-min_ idt 0. 70-min_cov 4-local_match_count_threshold 2-max_n_read 100-n_core 6", overlap_filtering_setting="-max_diff 40-max_cov 60-min_cov 2-bestn 10". The FALCON assembly was merged iteratively to exhaustion using batchhm. pl, as with version 1. 1 of the H. melpomene genome (see previous section).
Offspring were separated by barcode using process_radtags from version 1. 30 of Stacks  (Catchen et al. 2011). Parents and offspring were aligned to Hmel2 using the same alignment pipeline described above except using GATK version 3. 4-0, and Picard tools version 1. 135. UnifiedGenotyper was used for SNP calling rather than HaplotypeCaller, as HaplotypeCaller does not perform well with RAD sequencing data. SNPs where the father was homozygous, the mother was heterozygous (or, for the Z chromosome, had a different allele to the father), and the offspring all had genotypes, were identified. The resulting segregation patterns were sorted by number of SNPs. The most common segregation patterns and mirrors of these patterns were identified as chromosome prints, as no other patterns appeared at large numbers of SNPs, except for where all offspring were homozygous, or where the patterns were genotyping errors from the chromosome prints.
(2014)   for D. plexippus version 3 (http://monarchbase. umassmed. edu/ download/Dp_genome_v3. fasta. gz), P. polytes (http://papilio. nig. ac. jp/data/Ppolytes_genome. fa. gz), and P. xuthus (http://papilio. nig. ac. jp/data/Pxuthus_genome. fa. gz). Summary statistics were calculated using summarizeAssembly. py in PBSuite 14. 9. 9 (http:// sourceforge. net/projects/pb-jelly/; English et al. 2012) , and bespoke script genome_kb_plot. pl, used to calculate N50 values and make plots of number of scaffolds against cumulative genome length. BUSCO values were calculated using BUSCO v1. 1b1 with the set of 2675 arthropod genes  (Simão et al. 2015)  using generic Augustus parameters.
Genome size estimation from read alignments To estimate the number of true bases in the genome, we followed  Warr et al. (2015)  to calculate GC-content-adjusted read depths in 1-kb windows across Hmel2 (details on commands and scripts used can be found in the Dryad and GitHub repositories). BED files containing scaffold positions and gap positions were constructed with Unix tools operating on the Hmel2 scaffold, and chromosome AGP files (in the Hmel2 distribution). Reads for F1 father were aligned to Hmel2 using the alignment pipeline described above (Stampy, MarkDuplicates, IndelRealigner). Windows of 1 kb were constructed using BEDTools makewindows [using BEDTools v2. 25. 0 (Quinlan 2014)]; windows containing gaps were removed using BEDTools intersect, and per-base read coverage across Hmel2 for the F1 father was calculated with BEDTools genomecov using the -d option.
Median read depth and GC content was calculated for each window using bespoke script calculate_read_depth_gc_windows. py, ignoring windows shorter than 1 kb. The bespoke script adjust_read_ depth_windows. py was then used to adjust read depth for each 1-kb window w by a multiplying factor f, with f equal to the ratio of the overall median read depth across all windows, divided by the median read depth of all windows with the same GC percentage as window w. The same script estimates genome size as the sum across all windows of the number of bases in each window w multiplied by the GC-adjusted median read depth for w divided by the genome-wide median read depth.
Data availability The Hmel2 genome is available from LepBase v1. 0 (http://ensembl. lepbase. org). A distribution containing the genome and many supplementary files is available from http://butterflygenome. org. Sequence reads from the H. melpomene and E. isabella crosses are available from the European Nucleotide Archive (ENA), accession PRJEB11288. Pacific Biosciences data are available from ENA accession ERP005954. All bespoke code is available in File S3, and on GitHub at https://github. com/johnomics/Heliconius_melpomene_version_2. A Dryad repository containing the Hmel2 distribution, a frozen version of the GitHub repository, VCF files for the H. melpomene and E. isabella crosses, marker databases, GC content and read depths for 1-kb windows, and intermediate genome versions for Hmel1-1 and the PacBio assemblies is available at http://dx.
1 with haplotypes" and "PacBio FALCON" in Table  1  and Figure  1 ). We expect that increasing this coverage could have produced a very high quality genome with no additional data. However, this does not deal with heterozygosity across the genome and the resulting generation of many haplotypic scaffolds-a problem for most species and particularly for insects  (Richards and Murali 2015). As sequencing methods improve and true haplotypes can be assembled, it is hoped that full diploid genomes can be produced, and several efforts are already moving toward this  (Church et al. 2015 ; https://github. com/ekg/vg). We hope that, in the near future, it will be possible to assemble a diploid reference graph for H. melpomene, perhaps with the haplotypes reported here. However, as we wanted to preserve contiguity with Hmel1.
1, which was already a composite of both haplotypes, Hmel2 remains a composite haploid genome. HaploMerger has proved to be a very versatile assembly tool. In addition to having many options for varying the merging process, and for manually accepting or rejecting merges, HaploMerger is almost unique among similar tools in reporting where it has placed parts of the original genome in the new genome. This has allowed us to write scripts to transfer linkage map information and genes to new genome versions directly and automatically, without having to map the original genome scaffolds to the new genome separately and possibly erroneously (although we have used this approach to map genes that could not be transferred directly). We could then accept or reject merges where  they introduced misassemblies that conflicted with the linkage map or broke genes, and iterate the use of HaploMerger to collapse as many scaffolds as possible.
To achieve this goal, AGeCEP provides a language-agnostic extensible representation of queries and of reconfiguration actions that specify how queries can be transformed. Once an algorithm is expressed by this formalism, it can be integrated into diverse modern cloud-based CEP/SP systems and applied to an extensible set of query operators. In this way, AGeCEP overcomes the specificity of current CEP/SP research and enables creation of universal algorithms and strategies. More specifically, this research develops the following contributions: -The query lifecycle management (QLM) concept as a composition of four main steps: single-query optimization, multi-query optimization, operator placement, and runtime management; -A novel classification of query operators focused on their reconfiguration capabilities; -AGeCEP, the formalism used to represent CEP/SP queries and reconfiguration actions in a way that is both language-agnostic and appropriate for query management.
Complex Event Processing The basis of CEP was established by the work of  Luckham [2002]. At about the same time, the database community developed the first classical SP systems, such as Aurora  [Abadi et al. 2003]  and STREAM  [Arasu et al. 2004 ]. CEP and SP technologies share related goals, as both are concerned with processing continuous data flows from distributed sources to obtain timely responses to queries  [Cugola and Margara 2012]. Much of the early distributed CEP research was done in the context of publish/ subscribe systems, such as the Publish/Subscribe Applied to Distributed REsource Scheduling (PADRES) system  [Li and Jacobsen 2005]. Simultaneously, distributed SP research was under discussion, as in Aurora*  [Cherniack et al. 2003]  and Borealis  [Abadi et al. 2005]. Current research has obviously been influenced by these works, but the recent emergence of cloud computing has been strongly shaping the CEP/SP landscape as well.
More recently,  Rabinovich et al. [2010]  and  Weidlich et al. [2013]  built on their research by implementing simulation, static, and dynamic analysis of EPNs. EPNs share similarities with AGeCEP because they are also language agnostic and use directed graphs as their basic representation. However, the main goal of EPNs is to represent applications that can be translated into system-specific queries, whereas the proposed AGeCEP aims to provide a generic representation of queries. Cugola et al. [2015] , on the other hand, proposed an approach and an accompanying tool called CAVE that can be used to prove generic properties about user-defined queries by solving a constraint satisfiability problem. Finally,  Hong et al. [2009]  presented the work that most closely approximates the objectives of this research.
Query Definition Query definition is the step in which users define the CEP queries they want to execute. As mentioned in Section 2. 2, each system usually has its own query language that is used for this purpose. In addition, the way that users interact with the system to define and submit queries differs enormously from system to system. For example, commercial CEP packages such as Oracle Stream Explorer [Oracle 2015] and Software AG Apama  [Software AG 2015]  have full-fledged interfaces that help to define queries and monitor their execution. On the other hand, many academic  [Qian et al. 2013 ] and lower-level systems  [Storm 2015 ] provide only programming language Application Programming Interfaces (APIs) that are mostly targeted to software developers. Because the formalism presented in this research is independent of query language and, consequently, of how queries are defined, this step is not discussed further in the rest of this article.
Multi-Query Optimization Multi-query optimization (MQO) consists of finding overlaps (common partial results) between queries and merging them into a single query while maintaining their logical separation. This step usually optimizes the same criteria as the single-query optimization step. MQO can be executed as a separate step when a new query is created or periodically to take account of modifications in the underlying queries. In both cases, one of the greatest challenges is to decide which queries should be considered. MQO has received more attention than SQO. Early research was largely based on MQO for relational databases and focused on sharing specific operators, such as in  Madden et al. [2002] and Hammad et al. [2003]. Hong et al. [2009]  was one of the first work that formalized MQO as an independent task that can be applied to continuous queries.
ATTRIBUTED GRAPH REWRITING FOR CEP MANAGEMENT The AGeCEP formalism has been developed to enable specification of self-management policies that can be applied to query lifecycle management. To achieve this goal, two main challenges have to be overcome: The first is to find a query representation that is language agnostic, yet can encode all information required by self-management policies. The second is to find a way to specify unambiguous reconfiguration actions that act on the represented queries. The following subsections discuss these challenges further.
To identify these properties, a novel classification of query operators focusing on their reconfiguration capabilities was elaborated. Integrating a new operator into AGeCEP, therefore, is simply equivalent to classifying it properly. Details of this classification are presented in Section 5. Use of ADAG as a language agnostic representation of CEP queries is a natural choice corroborated by many studies in the literature. For instance, most CEP systems based on imperative languages use (non-attributed) DAGs to represent user queries  [Abadi et al. 2003; Storm 2015]. Systems that use declarative languages, on the other hand, transform user queries into query plans to make them "executable," which often leads to DAG-like structures (e. g. , the STREAM system and the CQL language  [Arasu et al.
Discussion In the context of self-management policies and autonomic computing, AGeCEP queries and reconfiguration actions are part of the knowledge base (KB). Specifically, AGeCEP focuses on representing "what can be done" and not on the decision-making process that determines "what should be done. " The MAPE-K modules are expected to use AGeCEP to implement their functions in conjunction with other information available in the KB such as monitored events, inference rules, and runtime environment models. Note that by limiting AGeCEP scope to queries and reconfiguration actions, it is possible to integrate AGeCEP with existing models and techniques rather than forcing the adoption of particular ones. By doing so, AGeCEP can be applied to a broader range of scenarios. Section 7 shows how existing representations and meta-models can be coupled with AGeCEP to cover the whole MAPE-K loop and thereby implement a complete autonomic manager.
Furthermore, it should be emphasized that neither vertex nor edge attributes are closed sets and can be extended whenever necessary. In particular, extrinsic properties such as operator placement and runtime information can also be modelled as vertex and edge attributes. 6. 1. 3. Example 1: Imperative Language. Figure  8  shows two queries q 1 and q 2 using the AGeCEP representation. To simplify the figure, some attributes have been omitted. The notation op specifies that the vertex is of the operator stereotype, whereas prod and cons qualify an event producer or an event consumer. These queries have been extracted from the Powersmiths' WOW system  [Powersmiths 2015 ], a sustainability management platform that uses live measurements of buildings to support energy management and education. In WOW, queries are implemented in Apache  Storm [2015]  and are used to process readings coming from building sensors managed by the platform.
The conversion from Storm queries to AGeCEP is straightforward because Storm also represents queries using DAGs. Query q 1 in Figure  8 (a) is used to convert readings from JSON to the native WOW format (XML). The query is implemented as a sequence of four operators: first, operator j 1 converts the JSON reading into a Java object. Then filters f 1 and f 2 remove invalid readings from the event stream. Finally, operator xml 1 converts the reading to an XML document and forwards it to the appropriate service. Query q 2 (Figure  8(b) ) is used for the same purpose but has a different structure. The query has two producers and two instances of operator f 12 , which executes a processing (filtering) logic equivalent to the sequential application of f 1 and f 2. 6. 1. 4. Example 2: Declarative Language.
6. 1. 5. Example 3: Pattern-Based Language. Figure  10  shows the conversion from a Cayuga Event Language (CEL)  [Demers et al. 2007 ] query to AGeCEP. CEL is a pattern-based language, even though it uses keywords that are similar to SQL. For instance, CEL uses the operators NEXT and F OLD to search for sequence of events that satisfy a stated condition, which is a construct characteristic of this language group. In Cayuga, queries are transformed into a non-deterministic finite state automaton to be executed. Figure  10(a)  shows a query q and its corresponding automaton. Even though this automaton can be represented as a graph, its semantics differs from AGe-CEP queries. For instance, the automaton states (vertices) are associated with input streams, whereas in AGeCEP vertices represent operators.
Hong et al. [2009]  presented a procedure to transform Cayuga automata to graphbased execution plans. Basically, they introduced two new query operators that implement the NEXT and F OLD logics and a procedure to convert edge transitions to a sequence of a filter followed by a projection. The execution plan for the example query is depicted in Figure  10(b). Once transformed to a graph-based execution plan, the conversion to AGeCEP is direct and results in the ADAG shown in Figure  10(c). 6. 1. 6. Discussion. The AGeCEP query representation has been designed to be as generic as possible. Most queries written in imperative and declarative languages can be converted directly to an AGeCEP ADAG. Pattern-based languages, on the other hand, require additional procedures for conversion, such as the one presented by  Hong et al.
[2009]  and demonstrated in the previous example. These additional procedures are needed because most pattern-based languages are executed as automata that do not follow AGeCEP graph-based model. In other words, there is a semantic mismatch between the models that must be solved before using AGeCEP to represent patternbased queries. Nevertheless, such mismatch should be solvable in most cases. For instance, Hong et al. [2009]  mentioned that the Sase language  [Wu et al. 2006 ] could be transformed to a graph-based execution plan using a procedure similar to that used to transform CEL queries. Similar procedures could also be applied to TESLA  [Cugola and Margara 2010]. The development of such procedures, however, is outside the scope of this article and may need to be analysed case by case.