dataesr/scientific-paragraphs-categorization huggingface dataset:
Expert Species Identification A total of 34,067 images were fully identified (i. e. , four classifications obtained) from the YPM Coretop Collection, of which 1,604 (4. 71% of the total) had 0% agreement (i. e. , all four identifiers disagreed on the species identification), 7,894 (23. 17%) had 50% agreement, 9,578 (28. 12%) had 75% agreement, and 14,991 (44. 00%) had 100% agreement. This resulted in a data set of 24,569 unique images (72. 12% of the total data set) with high-quality labels (i. e. , three or four of four identifiers agreed on the classification), representing 34 species. A human confusion matrix (Figure  2 ) was generated using the objects with high-quality labels by summing the number of identifications for each of the 34 species and determining the proportion of the identifications are correct or incorrect.
e. , the top-3 accuracy) and found a last run top 3 training accuracy of 100% and a top 3 validation accuracy of 97. 66%. Figure  3  shows the evolution of all these metrics through the 44 epochs that this final analysis was trained. The final model weights, along with the code used for training and validation, can be accessed at https://www. github. com/ahsiang/foram-classifier website. The machine confusion matrix is shown in Figure  4. Some differences in the general pattern of misidentifications between the machine and human confusion matrices present themselves. While the human misidentifications tend to be phylogenetically conservative (i. e. , when individuals are misidentified, they are usually misidentified as another species from the same genus), the misidentifications by the machine are more liberal-for instance, 21% of the validation specimens of Globigerinella calida are misidentified as Globigerina falconensis.
Moreover, technological advances and innovative workflows are allowing natural history museums to enter a new age of mass digitization of their collections (e. g. ,  Hudson et al. , 2015; Rillo et al. , 2016) , further contributing to the availability of abundant image data. Rapid, automated methods and pipelines such as the ones we describe here are a growing necessity not only as high-throughput imaging methods produce ever more data, but also in a world where rates of ecological turnover in the oceans are ever increasing as a result of a quickly changing environment. Zooniverse database and checked all segmented objects from Buckley data set. M. C. R. provided the original Buckley data set, performed formatting changes of the Buckley images for compatibility with Endless Forams database (endlessforams.
org), checked all resulting segmented Buckley images, and participated in identifying objects on the Zooniverse platform. M. J. M. V. , S. C. , C. V. D. , S. L. , A. J. , M. J. H. , B. M. , I. S. F. , B. W. , L. F. , J. M. , C. V. D. , U. B. , J. G. , K. M. E. , A. M. , and T. A. all identified at least 5,000 objects on the Zooniverse platform, in order of most to fewest identifications. H. D. led initial classification exploration via images, which led the project to the Zooniverse platform and community approach, and contributed classifications to the Zooniverse platform. G. M. provided original Buckley metadata and performed compatibility formatting changes of the Buckley images for Endless Forams database (endlessforams. org). N. R. developed and curates Endless Forams database (endlessforams.
Subject area Genetics and Transcriptomics More specific subject area Transcriptomics of Myrtaceae species Type of data Table, figure How data was acquired Leaves of individual plant from three endemic Myrtaceae species from New Caledonia were sampled for total RNA extraction. Pairedend library were prepared and RNA-Sequencing was performed by the Illumina HiSeq™ 2500 system. The obtained data was subjected to 1) de novo transcriptome assembly, 2) alignment to a reference genome and de novo transcriptome assembly, 3) differential gene expression and 4) variant calling analysis.
Data source location The nursery was located in Farino, South Province, New Caledonia (Long 165. 772024:, Lat: -21. 663800). The protected reserve was located in Bois du Sud, South Province, New Caledonia (Long 166. 758640:, Lat: -22. 169974) Data accessibility All raw data for Arillastrum gummiferum, Syzygium longifolium and Tristaniopsis glauca and the processed data (de novo transcriptome assemblies, transcriptome annotations and differential gene expression files) obtained in this study were deposited in the National Center for Biotechnology Information (NCBI) Gene Expression Omnibus (GEO) with the Superseries accession number GSE106750 and the subseries accession numbers GSE106735, GSE106736, GSE106738, GSE106740, GSE106741, GSE106746, GSE106747 and GSE106749. The Superseries is available at https://www.
ncbi. nlm. nih. gov/geo/ query/acc. cgi?acc ¼GSE106750 The VCF files can be downloaded at http://myrtaceae-omics. south green. fr/node/8 provided you are logged and have accepted the Terms of Use of these data. Related research article Soewarto J, Carriconde F, Hugot N, Bocs S, Hamelin C, Maggia L. Impact of Austropuccinia psidii in New Caledonia, a biodiversity hotspot. Forest Pathology. 2018;48  (2). doi:10. 1111/efp. 12402.
Raw reads cleaning RNA-sequencing for the three species, produced a total of 668,165,595 raw reads. The quality of paired-end raw reads in fastq format was assessed using FastQC sofware (http://www. bioinformatics. babraham. ac. uk/projects/fastqc/) with the script arcad_hts_0_fastqc_in_chains. pl. Raw reads were then processed with Cutadapt  [5]  using the script arcad_hts_1_cutadapt_in_chain. pl and the TruSeq index sequences corresponding to the samples. We also used Cutadapt to improve mean reads quality by trimming the start and the end of each reads. We then filtered the reads on the basis of their mean quality score, keeping those with a mean quality higher than 30 using arcad_hts_2_Filter_Fast-q_On_Mean_Quality. pl. Reads with length inferior to 35 bp were discarded. Thereafter, single reads  (i.
e. those for which the mate pair was discarded in the previous steps) were separated from pairedreads using arcad_hts_3_synchronized_paired_fastq_end. pl. Each processing steps for reads cleaning were followed by a FastQC quality control of reads (arcad_hts_0_fastqc_in_chains. pl). Each preprocessing cleaning steps is illustrated in Fig. 2. After the cleaning stage, we kept 96% of the initial reads (644,707,222 reads). An example of the efficiency of quality control on our data is shown in Fig. 3. A summary of the RNA-seq raw and clean data is presented in    [6]. The aligned data were passed to StringTie  [7]  for transcript assembly. A reference genome annotation file in GFF3 format (intron/exon positions) was provided to guide the transcripts assembly. The resulting binary files (. bam) were then filtered on quality using SAMtools view (with parameters -q 0 -F 4) to remove unmapped and multimapped reads  [8].
Then, we used the GATK tool SplitNCigarReads (with parameters -RMQF 255 -RMQT 60 -U ALLOW_N_CIGAR_READS), which splits reads into exon segment and hardclip any sequences overhanging into the intronic regions. The reads were realigned around INDELS and base quality values were recalibrated using RealignerTargetCreator and IndelRealigner GATK tools. Once the reads were pre-processed with Picard and GATK tools, variant calling was undertaken by two programs: Haplotype Caller (GATK) and an in-house script (Martin et al. , in prep. , Baurens et al. , in prep. ). Contrary to GATK, which uses statistics based on population genetics (which is not the case here), the in-house program only count the number of reads supporting each bases (A, T, G, C) at each covered sites for each accession. Based on this base count for each accession, a genotype was emitted based on a binomial test.
We obtained between 3 and 5 million of SNPs with Haplotype Caller for each species, and the in-house script identified between 8 and 11 million of SNPs (Supplementary Table  3 ). Several filtering criteria were used to exclude the less reliable SNPs from all the raw called variants dataset (Fig. 4 ). Because in-house SNP calling method implied to detect any difference between the reads and the reference (including false positive SNP), the initial outputs were too large and must be pre-filtered in order to allow comparison with those resulting from GATK SNP calling method. Firstly we applied an in-house pre-filter script (VcfPreFilter. 1. 0. py) on SNPs resulting from the in-house SNP calling method in order to remove SNPs due to sequencing errors. This pre-filter included the following parameters: a minimum site coverage (5 reads), a maximal site coverage (1000 reads) per SNP, a minimum allele frequency (0.
01) and a minimum allele coverage (2 reads per allele). Then for all datasets (from GATK and in-house SNP calling methods) we used an in-house VCF filter script (VCF_Filter. py) to keep only the more robust SNPs according to the following criteria: a minimum coverage  (10) , a maximal coverage (1000), a minimum frequency (0. 05), a minimum allele coverage (3) to keep a genotype; and no missing data were allowed to keep a variant site. Another in-house script was then used to remove monomorphic and multiallelic variants. And finally, as recommended by GATK, we used the Variant Filtration program to remove clustered SNP: in a window size of 10 pb we considered that 3 SNPS constitute a cluster. The final SNP set comprised the identified SNPs that had passed all filters. We compared the overlap among SNP positions obtained for each calling methods using vcf-compare from VCF tools  [10]  (Table  4  ).
De novo method 2. 5. 1. De novo transcriptome assembly We chose the individual with the highest read number after cleaning for each species for the de novo assembly as described by Sarah et al. [11]. De novo transcriptome assembly was carried out using Trinity software with 50 GB of memory (Inchworm, Chrysalis, and Butterfly modules)  [12]. Briefly, overlapping k-mers are extracted from the cleaned paired-reads. Inchworm module assembled the reads into contigs. Next, Chrysalis module ranked Inchworm contigs into clusters and constructed complete de Bruijn graphs for each cluster. Finally, Butterfly module processed the individual graphs in parallel to reconstruct transcript sequences in a manner that reflects the original cDNA molecules. To avoid redundant transcripts, we kept the longest isoform for each "trinity gene".
Assembly statistics (N50, contig length, GC content, etc) were computed by TrinityStats. pl embedded in Trinity (Table  5 ; Supplementary table 2). Transcriptome assembly for A. gummiferum resulted in 117 839 putative transcripts with an average contig length of 501 bp and N 50 of 1378 base pairs. Transcriptome assembly for S. longifolium yielded a total number of 89,782 putative transcripts with an average contig length of 530 bp in length and N 50 of 1406 base pairs. Transcriptome assembly for T. glauca-FAR resulted in 108,823 putative transcripts, with an average contig length of 547 bp and N 50 of 1396 base pairs. And finally, de novo transcriptome assembly for T. glauca-BDS resulted in 74,684 putative transcripts with an average contig length of 525 bp in length and N 50 of 1315 base pairs.
Aligning to the de novo transcriptomes We used the Burrows-Wheeler alignment tool (BWA-MEM) to map the cleaned reads from the 24 samples to the corresponding de novo transcriptome assemblies of each species  [13]. The read aligner, BWA-MEM, aligns each mate of a paired-end read at the same time and produces SAM/BAM files containing the alignments. Samtools view from the SAMtools 1. 3 package (http://www. htslib. org/doc/ samtools-1. 1. html) was used to sort and index the BAM files by coordinate and remove multimapped reads (parameters -q 0 -F 4). Alignment statistics for the mapping to de novo transcriptome assembles are displayed in Table  3.
Calling SNPs using de novo transcriptome assemblies The analysis pipeline we used is illustrated in Fig. 4. We processed the aligned reads to a cleaning process with the Galaxy  [14]  instance of the South Green platform http://galaxy. southgreen. fr/galaxy/. Based upon established GATK best practices, we used Realigner targer creator and IndelRealigner programs (LOD ¼ 5) to fix the misalignments due to the mapping process and perform a local realignment near the INDELS  [15, 16]. Variant calling was launched both with Unified Genotyper (GATK), using a minimum phred-scaled confidence threshold of 30  [6] , and an in-house scripts  (Martin et al. , in prep. , Baurens et al. , in prep. ). We obtained between 600 and 900 thousands of SNPs with Unified Genotyper for each species, and the in-house script identified between 9 and 12 millions of SNPs (Supplementary Table  4 ).
Variant filtering from mapping to the de novo transcriptomes We applied the same filtering process as explained earlier with the variant filtering from mapping to E. grandis genome. We compared the overlap among SNP positions obtained for each calling methods using vcf-compare from VCFtools  [10]  (Table  6 ). With de novo transcriptomes as reference for mapping, we found around 2=3 of the SNPs identified by Unified Genotyper (GATK) and in-house methods sharing identical positions on A. gummiferum and S. longifolium (Table  6 ). The proportion of identical SNPs positions is even bigger with T. glauca-BDS and T. glauca-FAR, up to 96% using the inhouse-script and 83% using Unified Genotyper (Table  6 ).
Annotation We ran FrameDP V1. 2. 2 software with default parameters  [20] , for the prediction of coding regions in the unigenes using the E. grandis protein database from Universal Protein Resource (UniProt-Swissprot). FrameDP V1. 2. 2 software automatically reverse-complement the sequences, however all the previous analysis were performed on the initial DNA strand orientation. Therefore, we changed the reverse-complemented orientation from FrameDP results to the initial genes orientation. The polypeptides sequences obtained were submitted to the Basic Local Alignment Search Tool (BLAST) to search against homologs in the UnitProt (Swiss-Prot and Trembl) databases. We identified 45,517 protein-coding genes in T. glauca-BDS, 50,173 protein-coding genes in S. longifolium, 57,366 proteincoding genes in T.
Genome browser To make easier the analysis of the massive amounts of genetic information that was generated during this study, we used the JBrowse tool from GMOD project (Generic Model Organism Database project)  [21]. JBrowse is a web-based genome browser, allowing interactively visualizing and exploring a large genomic dataset  [22]. Each track consists of a particular type of sequence feature along a reference sequence, as showed in Fig. 8. We deployed five different JBrowses with the data from each species, taking first the E. grandis genome as a reference for alignment, then the de novo transcriptomes (A. gummiferum, S. longifolium, T. glauca-FAR and T. glauca-BDS). To build the JBrowses, we rely on a released workflow  [23]  and we imported the following files: reference sequence files (de novo transcriptomes (.
faa) or E. grandis reference genome (. fna), alignments files (. bam), variants calling files (. vcf), annotations files (. gff3), and EdgeR output files containing the RPKM (expression level) of each locus for the samples (. tsv). To  load the following data we used the following perl scripts provided by the JBrowse developers: prepare-refseqs. pl for fasta files and flatfile-to-json. pl for. gff3 and. tsv files. To load the others data types we had to edit two files tracks. conf and trackList. json, respectively displaying the dataset-specific names and configuration. We also used a JBrowse plugin called Multi-VariantViewer (multivariantviewerjbrowsepluging) to display the corresponding genotype for each SNP from each individual of this study (https://github. com/elsiklab/multivariantviewer).
Acknowledgements The authors acknowledge the  Plateforme du Vivant (Nouméa, New Caledonia ) and the Grand plateau technique regional de génotypage ( CIRAD, UMR AGAP, Montpellier, France ) for their facilities accesses. The authors acknowledge the  South Green Platform  (http://www. southgreen. fr)for providing computational resources (Galaxy and  CIRAD UMR-AGAP HPC ) that have contributed to the research results reported within this paper. This work was supported financially by  GLENCORE society, Agronomic Institute of New Caledonia (IAC)  and  CIRAD (centre de Coopération Internationale en Recherche Agronomique pour le Développement). Thanks to  A. Rey  from the  legal department of CIRAD  for her help in the drafting of the  Terms of Use  of the data available for downloading on http:// myrtaceae-omics.
Human whole-genome-sequencing reveals about 4 000 000 genomic variants per individual. These data are mostly stored as VCF-format files. Although many variant analysis methods accept VCF as input, many other tools require DNA or protein sequences, particularly for splicing prediction, sequence alignment, phylogenetic analysis, and structure prediction. However, there is no existing webserver capable of extracting DNA/protein sequences for genomic variants from VCF files in a user-friendly and efficient manner. We developed the SeqTailor webserver to bridge this gap, by enabling rapid extraction of (i) DNA sequences around genomic variants, with customizable window sizes and options to annotate the splice sites closest to the variants and to consider the neighboring variants within the window; and (ii) protein sequences encoded by the DNA sequences around genomic variants, with built-in SnpEff annotator and customizable window sizes.
SeqTailor supports 11 species, including: human (GRCh37/GRCh38), chimpanzee, mouse, rat, cow, chicken, lizard, zebrafish, fruitfly, Arabidopsis and rice. Standalone programs are provided for command-line-based needs. SeqTailor streamlines the sequence extraction process, and accelerates the analysis of genomic variants with software requiring DNA/protein sequences. It will facilitate the study of genomic variation, by increasing the feasibility of sequence-based analysis and prediction.
INTRODUCTION In humans, whole-genome sequencing (WGS) and wholeexome sequencing (WES) generate large amounts of genomic data, generally revealing for each individual studied about 4 000 000 variants by WGS and 80 000 variants by WES, with both identifying about 20 000 coding variants. According to the 125 748 WES data released by gnomAD database  (1) , there are ∼40% variants are nonsynonymous (missense, stop-gained, stop-lost, start-lost, and frameshift), 17% are synonymous, 35% are intronic variants, whereas ∼5% of the WES variants are in the essential splice sites. These variant data are usually recorded in variant call format (VCF), which includes structured fields for chromosome (CHROM), position in reference genome sequence (POS), reference allele (REF), alternative allele (ALT) and others (e.
g. QUAL, FILTER, INFO, etc. ). The major human genomic variation databases (e. g. gnomAD/ExAC (1), 1000 Genomes Project (2)), the inherited disease mutation databases (e. g. HGMD  (3) , and Clin-Var (4)), and most research laboratories and publications present and manage genomic variants in VCF format. For further variant annotations, predictions and visualizations, most of the leading tools use VCF as input (e. g. SnpEff  (5) , VEP  (6) , ANNOVAR  (7) , CADD  (8)  and PopViz  (9) ). Research, 2019, Vol. 47 , Web Server issue However, as the DNA sequence around the variant site presents richer information than the variant alone, many popular genomic software require DNA sequences as input, particularly for splicing prediction (e. g. Human Splicing Finder  (10) , NetGene2  (11) , Spliceman  (12) ), sequence homology search (e.
W624 Nucleic Acids Users wishing to analyze genomic variation data in VCF files with software that requires DNA or protein sequence must therefore extract the corresponding reference and alternative sequences. However, the available tools (e. g. UCSC genome browser  (25) , BEDTools-getfasta  (26) , Samtools-faidx  (27) , IGV  (28) ) can be used to extract the reference sequences, but not to generate the mutated alternative sequences. UCSC genome browser provides web interface with pages of user-defined parameters, BEDTools and samtools require script knowledge, while IGV is impractical for high-throughput applications (see detailed comparison in Discussion). Thus, there is currently no tool available for simple extraction of reference and alternative DNA/protein sequences directly from VCF files.
Webserver The SeqTailor webserver is run by Apache HTTP (version 2. 2. 15), on a Red Hat Enterprise Linux Server (version 6. 9), with 8 Intel CPU processors@2. 4 GHz and 48GB RAM. The website interface is designed and presented in HTML, PHP, CSS and JavaScript. The data are stored in MySQL tables and plain text files. The sequence extraction programs are coded in Python 2. 7.
Data collection and pre-processing The reference genome sequences, gene annotations, coding sequences (CDS) and protein sequences for 11 species (human (GRCh37/GRCh38), chimpanzee, mouse, rat, cow, chicken, lizard, zebrafish, fruit fly, Arabidopsis and rice) were collected from Ensembl Database release 95  (29). We extracted the reference genome sequences of autosomal chromosomes, sex chromosomes and mitochondria for Se-qTailor DNA sequence extraction. In addition, since the key feature of human GRCh38 assembly is to provide alternate loci for a more robust representation of human population variations (including the highly variant MHC region and the divergent haplotypes (30)), we also supported DNA sequence extraction from 329 alternate loci and scaffolds in GRCh38 assembly (Supplementary Table  S1 ) according to the Ensembl Database.
Moreover, we extracted the transcripts of 15 biotypes in the categories of protein coding and pseudogenes (Supplementary Table  S2 ) to compose a collection of 'all transcripts' in SeqTailor. We further selected the transcripts labeled as protein-coding biotype only, and identified the longest protein-coding transcript for each gene, to compose a collection of 'canonical transcripts'. The genomic positions of all exons were extracted, and hence the donor splice sites (5 end of the intron) and the acceptors splice sites (3 end of the intron) were obtained and used to annotate the nearest splice sites for genomic variants. The workflow of data collection and pre-processing is shown in Figure  1 , and the supported 11 species are summarized in Table  1.
Built-in SnpEff variant annotation SnpEff v4. 3 was integrated in SeqTailor, for variant annotation and consequence prediction  (5) , with multi-threading configuration. Based on the previously identified collections of all transcripts and canonical transcripts, we extracted the corresponding gene annotations, CDS sequences and protein sequences respectively. These data were then employed to build the customized SnpEff databases on all transcripts and canonical transcripts separately. The customized SnpEff database of canonical transcripts allows 3-4× faster variant annotation, compared to annotating variants on all transcripts. The genomic variants are then filtered to retain 13 types of the consequences, which have direct impact on protein sequences, including: missense, stop-gained, synonymous, frameshift, inframe insertion/deletion, conservative/disruptive in-frame insertion/deletion, and feature/transcript/gene ablation.
Runtime evaluation The runtime evaluation was performed by applying four input data sizes (10, 100, 1000 and 10 000 genomic variants) on human reference genome GRCh37. We generated 50 VCF files for each input data size, by randomly selecting variants from the gnomAD database (1), and retaining only the first five columns of information (CHROM, POS, ID, REF, ALT). We therefore used a total of 200 VCF files with different numbers of human genomic variants to evaluate the runtime performance of SeqTailor for DNA and protein sequence extraction.