dataesr/scientific-paragraphs-categorization huggingface dataset:
Initial analysis of micrographs and segments The EM operator records electron micrographs in several different ways. Currently, film, CCD cameras and direct detectors are the common sources of EM data. After film has been digitized all types of data are available as images in various formats. SPRING accepts all the formats of micrograph data that EMAN2 currently supports such as standard MRC, IMAGIC, SPIDER, TIF formats. MICEXAM examines the micrographs by analyzing the power spectra tiles to exclude images that suffer from poor information transfer at higher resolutions due to charging or drift  (Hohn et al. , 2007). MIC-CTFDETERMINE determines the CTF of the micrographs by interfacing with CTFFIND initially and optionally refines parameters using CTFTILT  (Mindell and Grigorieff, 2003). The program captures a reduced output of CTFFIND and CTFTILT and the results are stored in the SPRING database to be retrieved for further processing.
After the selection of high-quality micrographs, helices need to be extracted from the images. For this purpose, the helices are interactively picked using external programs. In the past, EMAN's HELIXBOXER or BOXER with the helix option was used  (Ludtke et al. , 1999 ) (EMAN2 has an updated version named E2HELIX-BOXER). BSOFT is also capable of picking filaments with significant curvature and recording their helix paths  (Heymann and Belnap, 2007). The program SEGMENT from SPRING extracts a complete data set of overlapping segments using the provided coordinates from either EMAN, EMAN2 or BSOFT, applies CTF correction by either phase-flipping or convolving the segments with the determined CTF and stores coordinates and the derived in-plane rotation in a database file. For further 2D analysis, the stack of inplane rotated segments is analyzed by adding their power spectra using SEGMENTEXAM.
The presence or absence of layer lines is a good indicator of the degree of order and symmetry of the specimen. In particular, the reduction of the power spectrum into a ''collapsed'' one-dimensional profile by averaging the spectrum along the layer lines is useful to assess the existence of weak layer lines. Furthermore, width profiles of segments allow a quick characterization of the specimen. For further characterization, SEGMENT-CLASS iteratively classifies and aligns the segment stack using a k-means clustering algorithm from SPARX  (Hohn et al. , 2007)  (Fig. 3A-D ). Segment assignment to classes is recorded in a database and can be used as a selection criterion at a later stage during the 3D structure refinement. The resulting classes can be further analyzed in real space and in the Fourier domain.
Program Description Micrograph analysis (Springmicrograph) Micexam Program to examine micrograph quality by computing a localized power spectrum using EMAN2's e2scaneval. py and an averaged power spectrum from overlapping tiles using SPARX' sx_welch_pw2. py Micctfdetermine Program to determine CTF parameters from a set of micrographs using CTFFIND and CTFTILT  (Mindell and Grigorieff, 2003 ) Segment analysis (Spring2d).
Segment Program to extract overlapping segments from micrographs Segmentexam Program to examine all of excised in-plane rotated segments and compute their collapsed (1D) and 2D power spectrum and width profile of helices Segmentclass Program to classify excised in-plane rotated segments using SPARX's k-means clustering Segclassexam Program to examine helix classes to compute their collapsed (1D) and 2D power spectrum and width profile of helices Segclasslayer Program to extract amplitudes and phases from desired layer lines of class averages Seglayer2lattice Program to simulate helical diffraction pattern and plot helical lattice from a series of indexed layer lines or rise/rotation parameters Segmentplot Program to plot parameters from segmented helices Segment 3D reconstruction (Spring3d) Segclassreconstruct Program to compute 3D reconstruction from a single class average using a range of different helical symmetries Seggridexplore Program to interactively explore grid searches according to different criteria Segmentrefine3d Program to iteratively refine a 3D structure of helical specimens from segment stacks Segrefine3plot Program to plot refinement parameters from segmentrefine3d Segrefine3dgrid Program to optimize segmentrefine3d reconstruction by varying refinement parameters systematically on a grid Segrefine3dinspect Program to interactively inspect 3D reconstructions from segmentrefine3d performed with a different stack that was not segmented by a regular step size but by using a randomized step between 0 and the desired step size.
In some cases, indexing of a helical lattice may help to limit the number of solutions. In order to better access and evaluate the large grid of different symmetry parameters, we developed the interactive GUI-based SEGGRIDEX-PLORE program (Fig. 1D ) where multiple comparison criteria can be browsed simultaneously and the associated data files such as the simulated power spectrum, the corresponding experimental power spectrum and 3D reconstruction can be immediately opened by clicking on the respective grid point. This option of interactively browsing the results makes the evaluation of these many symmetry combinations easily accessible and user-friendly. To further test the applicability of the class-based reconstruction approach, we generated test classes from 3D reconstructions of the flagellar hook and Respiratory Syncytial Virus (RSV) nucleocapsid (EMDB 1647/ 1622, respectively)  (Fujii et al.
6 , PDB code 2OM3, 2TMV)  (Namba et al. , 1989; Sachse et al. , 2007)  and the outer densities in particular are best defined in comparison with the other 3D structures. It is important to note that refinement convergence and statistics such as FSC and azimuthal angle distribution do not reveal whether the symmetry is correct or wrong. Some solutions may even be interpreted as a fold containing a-helices but are clearly incompatible with the atomic reference structure. In conclusion, the aim remains to determine an EM-map at sufficient resolution to verify the 3D reconstruction and symmetry in order to recognize a characteristic fold or polypeptide backbone at medium to high resolution (4-10 Å) or an unambiguous fit of the PDB structure at lower resolution (>10 Å). To further refine the correct symmetry, we performed a symmetry grid search of iterative 3D refinements around 16.
High-resolution helical reconstruction Our previous description of single-particle helical based reconstruction for TMV was implemented as a prototype using SPIDER scripts  (Sachse et al. , 2007). The current article focuses on the bundling and extension of the previous approach into an efficient and   66, 14. 34, 15. 66, 16. 34, 17. 66 and 18. 34 subunits per turn, respectively. Only the correct reference structure with 16. 34 numbers of subunits per turn is compatible with the four-helix bundle atomic structure (PDB code 2OM3). reproducible image processing pipeline with significant optimizations (Fig. 2 ). In SPRING, we have re-implemented many of the features using the python API of SPARX/EMAN2. Briefly, we would like to summarize the important features of the structure refinement that we have implemented as previously described.
For a more detailed discussion of these topics consult our article  (Sachse et al. , 2007). Once the overlapping segments have been extracted the images are convolved with the corresponding CTF as estimated from the CTFTILT measurement and the position of the segment on the micrograph. The original in-plane rotation of the segments is retained in the images and recorded in a database. This step is critical for correcting astigmatism of the CTF on the segment level and beneficial to avoid multiple interpolation steps when segments are included in the 3D reconstruction. These segments are subjected to projection matching and followed by 3D reconstruction. The polarity assignments of segments within a helix are recorded and excluded if an opposite polarity has been found. Furthermore, alignment restraints are applied to exclude excessive shifts against a mean path of the helix.
First, seamless parallelization was implemented by Mpi4Py -a python interface to OpenMPI. Moreover, segments are downscaled internally in multiple resolution steps to reduce computational cost. The complete refinement is broken up into appropriate resolution aim ranges that the user is able to select (Table  3 ). The underlying principle is that a 3D reconstruction at the targeted resolution can be obtained using a pixel size where 3Â pixel size is assumed as the practical resolution limit due to interpolation errors. SPRING will determine the required decimation factor to achieve the respective resolution aim. Thus, the segments are low-pass filtered and subsequently decimated and as a result the convergence of the refinement is accelerated. By default, the refinement projects 90 projections around the helical axis and assumes seven possible out-of-plane tilt angles from À12°to 12°.
We, therefore, reconstruct a reduced size volume of helix width dimensions and for the subsequent step of projection helically expand the volume back to the intended dimension required for alignment. In order to further demonstrate the complete functionality of the package, we performed a structure refinement with a smaller subset of our previous data set used for the prototype of highresolution reconstruction. With a total of 50,000 asymmetric units extracted from two micrographs we computed a 3D reconstruction at 6. 4 Å resolution (Fig. 8A-F ). The final 3D structure was interactively B-factor sharpened by -150 1/(Å) 2 and filtered to a resolution cutoff of 5 Å using the program SEGREFINE3DINSPECT to correctly display the EM densities  (Rosenthal and Henderson, 2003). The total refinement time of 16 iterations from low to highest resolution took 45 h on 48 Intel Xeon 2.
4 GHz processors. Next, we wondered how the resolution of SPRING reconstructions compares with the previously published structure that was computed with SPIDER scripts  (Sachse et al. , 2007). Consequently, we determined a structure using the previous data set of 200,000 asymmetric units and measured very similar resolutions with cutoffs at 4. 8/ 4. 3 Å (FSC 0. 5/0. 143) when compared with the previously determined 4. 7/4. 3 Å (EMDB-1316) (Fig. 8F ). At closer inspection of the map, the EM densities provide a very similar level of detail for structure interpretation (Fig. 8H/I ). In order to increase the homogeneity of single-particle data sets, classification strategies are routinely employed that rely on principal component analyses  (van Heel and Frank, 1981)  or crosscorrelation based approaches.
In helical processing in particular, projection-matching based sorting of varying pitches has been applied to improve structural homogeneity of particular helical symmetries  (Wang et al. , 2006). In order to conveniently track the membership of segments to micrographs, helices, classes and other microscope-related parameters SPRING relies on an SQL database that records the available data of each segment. If desired, the user can apply segment exclusion criteria that are based on properties before any refinement is done, e. g. class assignment, micrograph, helix, defocus and astigmatism. In addition, segment exclusion can also be based on refinement parameters such as crosscorrelation peaks, out-of-plane tilt and excessive x-shifts. For further analysis, refinement parameters can easily be plotted as a function of distance along the helix using the program SEGRE-FINE3DPLOT.
Outlook The development of the here-described processing package SPRING will facilitate many of the steps of single-based particle helical reconstruction. One of the most time-consuming processing steps, however, will remain the determination of the helical symmetry parameters because validation is really only possible once the obtained resolution allows an unambiguous fit of a high-resolution PDB model. This ambiguity in 2D projections may be overcome by additional experimental approaches to evaluate the helical symmetry directly from 3D tomograms as was previously demonstrated on retroviral Mason-Pfizer Monkey virus tubes  (Bharat et al. , 2012). Those initial lattice parameters can be further refined using SPRING without the need of large-scale exploration and validation of symmetry parameters.
In conclusion, SPRING will significantly speed up helical structure determination by merging many processing steps into a single processing workflow. Furthermore, it will enable the user to address more challenging samples that exhibit significant heterogeneity in their helical symmetries or helical order. With the advent of direct electron detectors additional improvements in image quality can be expected and will increase the obtainable resolution of helical specimens with fewer asymmetric units and for example facilitate the processing of data sets of helices with different symmetry families. The SPRING website (http://www. sachse. embl. de/emspring) provides a tutorial on how to process the two described micrographs of the TMV data set used in this publication.
Background: Metagenomics, based on culture-independent sequencing, is a well-fitted approach to provide insights into the composition, structure and dynamics of environmental viral communities. Following recent advances in sequencing technologies, new challenges arise for existing bioinformatic tools dedicated to viral metagenome (i. e. virome) analysis as (i) the number of viromes is rapidly growing and (ii) large genomic fragments can now be obtained by assembling the huge amount of sequence data generated for each metagenome. Results: To face these challenges, a new version of Metavir was developed. First, all Metavir tools have been adapted to support comparative analysis of viromes in order to improve the analysis of multiple datasets. In addition to the sequence comparison previously provided, viromes can now be compared through their k-mer frequencies, their taxonomic compositions, recruitment plots and phylogenetic trees containing sequences from different datasets.
Two web-servers are currently available for a comprehensive virome analysis: Metavir  [24] , and Virome  [25]. A pipeline (the Viral Metagenome Affiliation Pipeline  [26] ) was also described but to our knowledge is not available neither as a standalone software or through a web page. Yet, none of these bioinformatic tools were designed for the analysis of assembled datasets and the absence of adapted tools for such assembled viromes was pinpointed as a major bottleneck for viral metagenomic studies  [25, 27]. Moreover, the growing number of generated viromes calls for the development of comparison strategies to go beyond individual analysis of each dataset. Here, we introduce a new version of Metavir that tackles these two limitations. Metavir 2 includes (i) new ways to compare datasets and (ii) a whole new section which forms the first tool designed for a comprehensive analysis of assembled virome sequences.
Input and metadata Registered users can upload their own sequence datasets, either short reads or assembled contigs, in a private space. Input data are checked for being only composed of DNA sequences in fasta format (compressed files in zip, gzip or tar. gz format are accepted). Due to the size of Illumina's raw datasets (~50 Gb) and computing time required for assembling each dataset, the assembly step cannot be computed through Metavir. Furthermore, a wide range of softwares are available for this step and the choice depends on the type of the sequencing and the nature of the sample: Newbler (454 Life Sciences) is the main software used so far for 454 data  [20, 28, 29] , and Illumina data can be assembled with Idba_ud  [15] , SOAP  [30] , MetaVelvet  [31]  or OptiDBA  [16]. A set of public viromes is also already available for users to compare with their dataset(s).
Taxonomic composition Virome reads are first compared to the complete viral genomes of the RefSeq Virus database using BLAST. The taxonomic composition is then determined using either raw number of best hits or number of best hits normalized by genome length using GAAS  [32]. Krona  [33]  is now used to generate interactive charts representing taxonomic composition of one or more viromes. A custom-designed javascript program has also been implemented to visualize these compositions as interactive heatmaps, with each column representing a dataset and each row a group of viral species. Columns can be switched by mouse drag and drop. Viral species are classified according to the up-to-date NCBI taxonomy, and viral groups can be folded and unfolded with a mouse click.
k-mer frequency bias A virome comparison based on k-mer frequency bias (di-, tri-and tetranucleotides are available) has been implemented as described by Willner and collaborators  [34]. Unlike the other available comparison method, based on sequence similarity (generated using reciprocal tBLASTx) and requiring datasets containing at least 50,000 sequences of 100bp, k-mer nucleotide frequencies can be computed for all datasets without size restriction. Briefly, k-mer frequency distribution bias are computed by a custom Perl script and then compared for each pair of viromes. Pairwise euclidian distances between viromes are stored in a matrix, which can be used as input either in a hierarchical clustering or a non-metric multidimensional scaling. Both analysis are computed with R  [35]  using pvclust  [36]  and vegan  [37]  libraries respectively.
Individual viral genome recruitment plots Using the best BLAST hit results against RefseqVirus, each virome sequence with a hit is affiliated to a unique viral genome, i. e. each read is recruited by a reference virus. For any selected viral genome, two types of recruitment plots are then available: (i) a scatter plot displaying each recruited read as a dot depending on the position on the genome (on the x-axis) and the identity percentage of the BLAST hit (on the y-axis), and (ii) an histogram presenting the number of recruited reads for each 500-nt long genome part. These plots are generated using the ggplot2 R library  [40]. Additional viromes that contain sequences recruited by the selected genomes are also listed and can be added to the current plot. When several datasets are selected, a color is attributed to each virome, used to color dots (in scatter plots) or stacked histograms (in histograms).
Contig annotation Open reading frames (ORFs) are first predicted for each contig through MetaGeneAnnotator  [41]. A custom Perl script was designed to detect circular contigs by looking for identical k-mer at the two ends of the sequences. Each circular contig is then trimmed to remove all redundant parts. In order to be able to predict genes spanning the origin of circular contigs, a temporary version of circular contigs is used in the ORF prediction software, in which the first 1,000 nucleotides are duplicated and added at the contig's end. It has to be noted that this detection of circular contigs will not be effective for contigs computed with assembler like Newbler which already detect and remove such similarity between contig ends. All predicted translated ORFs are then compared to several databases, namely the RefseqVirus protein database from the NCBI using BLASTp  [42] , with a threshold of 10 -3 on e-value, and the PFAM database of protein domains (version 26.
0;  [43] ) using HMMScan  [44] , with a threshold of 30 on score. A direct comparison of ORFs within a virome is also computed through a BLASTp with the same threshold of 10 -3 on e-value. The taxonomic composition and sequence diversity are not calculated the same way for datasets made of long genomic sequences compared to those made of short reads. Using the BLASTp results against reference viruses, three types of taxonomic compositions are computed for each dataset. These compositions are based on (i) best BLAST hit affiliation of each predicted gene, (ii) best BLAST hit affiliation of each contig, and (iii) lowest common ancestor affiliation of each contig. This LCA affiliation is designed to take into account the multiple hits on a single contig: up to five affiliated genes (if available) are considered for each contig, and the affiliation is made at the highest common taxonomy level of the best BLAST hit from these selected genes.
Results and graphics download All sequence datasets used in a Metavir analysis are available for download in fasta format (affiliated and uncharacterized sequences, sequences included in phylogenetic trees and sequences included in recruitment plots). All tables (taxonomic heatmap, contig and ORF affiliations, results for recruitment analysis) can be downloaded as csv files that can be imported in spreadsheet softwares. Contig annotations are available in GenBank file format, which can be used in many downstream tools like Artemis  [47]  or Easyfig  [48]. These GenBank files contain the lowest common ancestor affiliation of the contig, as well as the best BLAST hit affiliation of each ORF, the functional annotation of each ORF in PFAM domain, and the sequences of each predicted CDS. All interactive charts and pictures (contig maps, contig comparisons, phylogenetic trees) can be downloaded in svg format, a publication-ready vectorial format easy to modify using graphics softwares.
Case study: using metavir to analyze the human gut virome Two different datasets from the human gut viral community were chosen to illustrate the results that can be obtained with Metavir 2. First, a set of 16 viromes was used to illustrate the section dedicated to unassembled datasets (  [49] ; project "Human Gut Diet" on Metavir). These metagenomes, sequenced with 454 GS Titanium (884,628 reads of 350 bp/310 Mb), were initially designed to study the dynamics of human gut viral community during a perturbation by a dietary intervention. Two individuals were fed a high fat/low fiber diet (H1 and H2), three were fed a low fat/high fiber (L1, L2 and L3) and one was on an ad-lib diet (X). Samples were collected at up to four time points (days 1, 2, 7 and 8). The second dataset is an assembled virome, resulting from the assembly of Illumina Hi-Seq 2000 reads (5.
Results and discussion Metavir, a web server dedicated to the analysis of viromes uploaded by registered users, can now be used to analyze the two existing types of datasets: (i) viromes composed of raw reads, mostly generated using pyrosequencing technology and (ii) viromes assembled into contigs, a strategy possible with datasets sequenced with either pyrosequencing or Illumina technology. The novelties of version 2 of Metavir will be illustrated here using both types of datasets (unassembled 454 reads  [49]  and Illumina assembled contigs  [16] ), all from human gut samples.
Additions to the unassembled datasets section Most published viral metagenomes are still analyzed at the read level. Indeed, pyrosequencing technology is often chosen to generate viromes, as this technology produces long reads and several samples can be easily multiplexed in a single run. Thus, the number of reads in each multiplexed dataset is generally insufficient to produce an assembly. Furthermore, the multiple datasets generated make it possible to study spatial or temporal dynamics in environmental communities  [10, 22, [50] [51] [52]  or different individuals subjected to different conditions for eukaryote-associated viromes (e. g. different diets in  [49] ). In this context, the comparison of multiple datasets was our major focus while extending the section dedicated to unassembled datasets.