dataesr/scientific-paragraphs-categorization huggingface dataset:
META-ANALYSES OF PUBLIC INFERENCE METH-ODS As the benchmarking service requires every public method to make their inferences on a standardized dataset, the publicly available datasets can be easily used as a source for meta-prediction of orthologous relationships across multiple orthology inference methods. This has multiple advantages: distant orthology relations may not be unveiled by all inference methods, thus diversifying the source of prediction makes it more likely one could discover such pairs. On the other hand, if a user is interested in relying only on the most robust orthology inferences, the number of methods that infer any orthologous pair can be used as a confidence score. This aggregation of orthology inference methods is already made by several meta-prediction resources: HGNC Comparison of Orthology Predictions  (30, 31) , DIOPT  (32) , OrthoList  (33)  or WORMHOLE  (34).
However, for Domainoid and Sonic- Paranoid, this is affected by the fact that there is at least one other public prediction in the dataset based on ilar methodological premises. Accordingly, when we repeat this analysis by considering only one of the redundant sets, the number of 'shared pairs' from these two methods diminishes, getting them closer to PANTHER, OrthoFinder and OrthoMCL (Supplementary Figure  S1 ). Few methods provide a high number of pairs that are specific to their own methods, but interestingly and perhaps expectedly, the methods that provide the highest number of 'method-specific' predictions (Ensembl Compara and OMA GETHOGs) correspond to the methods that generally exhibit higher recall, at the cost of precision. Conversely, the methods that tend to strike a balance between both recall and precision mostly recover pairs that are predicted by at least one other method.
In addition, this comparison unveils expected similarities between some of the public methods included in the benchmarks: for example, all predictions made by PAN-THER LDO are made by PANTHER, and the same is true for PhylomeDB and MetaPhOrs, as well as for Inparanoid and Domainoid, meaning that in these cases the first set of predictions is a subset of the second. Similarly, the inference sets from Inparanoid, Domainoid, and SonicParanoid run with different parameters have high overlap (but not Hieranoid), as expected from methods based on very similar approaches. This surface analysis of the inference prediction methods is only an example of what the benchmarking service platform, and its freely available data from public methods, makes possible in terms of additional assessments of the inference set provided by every method on a common dataset.
PERSPECTIVES After >5 years of continued service, the Benchmarking platform continues to be widely used and plays its role in evaluating the performance of the existing orthology software and databases. This is made possible by a commitment from the community to provide regularly updated data (orthology prediction with different methods) and by the incremental improvement to the benchmark over the years. In this year's update, the improvement took the form of new benchmarks, an increasing traceability in the update of the reference proteomes, and easier accessibility of public data, which opens the path to more efficient data sharing outside of the community. The current Quest for Orthologs benchmark is designed to accommodate a variety of orthologous benchmarks, yielding a diversity of orthologous inferences.
Thus, the legacy benchmark data type has been the orthologous pairs: a type of prediction that can be made by all methods. Yet, most comparative genomics studies done today aim to consider orthologous relations across a whole set of species, at the gene families level, an endeavor that is only made possible by hierarchical orthologous groups and gene tree predictions. In the short term, we aim to start including a groupbased benchmark to the benchmarking service and, as a first example, we implemented a version of the OrthoBench benchmark  (35, 36)  to the benchmarking service. However, integrating group-based benchmarks into the benchmarking service necessitates solving additional challenges. First, we will work on an updated platform to be able to handle independently the universal pair-based benchmark and the narrower orthologous groups benchmark, which can be applied only to a subset of methods.
Though most methods perform well on the benchmark with high recall and precision, their relative ranking is indicative of how well they are able to solve the most difficult cases in comparison to others. The measures used to evaluate performance, especially accuracy, are non-trivial to implement because of the absence of 'negative set' in the dataset. Our choice here was to use a conservative definition of what is called a false prediction rather than risk wrongly penalizing a method for a real prediction. Thus, users are invited to refer to the relative ranking error interval rather than absolute value only when using the benchmark. The goal of the Benchmarking Service will be to continue, whenever possible, to integrate new benchmarks using curated data when it has the potential to add valuable information about the performance of the public methods on different conditions.
We invite the broader community to reach out to the consortium if they believe other dataset could be used for this endeavor in the future Despite there being several methods trying to provide orthology inference, the meta-analysis provided here shows that the agreement level between their predictions is not necessarily high. Partly, it comes down to the stated goal of the methods: at one end of the spectrum, some aim to provide fewer, high confidence orthology calls, while others aim to identify more orthologous relations with some impact on precision. Users should weigh these differences when selecting orthology inference methods, and select the one most adapted to their purpose. The different benchmarks provided by the orthology benchmark platform should give them additional information in which context it is optimal to use one method over another.
Here we present SMAP, a software package that implements a suite of computational tools to extract multiallelic haplotypes using read-backed haplotyping. SMAP tools first perform accurate read processing and analyze read mapping distributions across sample sets. Then, two complementary modules can be invoked for haplotype calling: SMAP haplotype-sites combines known Single Nucleotide Polymorphisms (SNPs) and/or read mapping position polymorphisms (SMAPs) to reconstruct compressed, read-referenceencoded haplotype strings. In contrast, SMAP haplotype-window works independent of prior knowledge of polymorphisms, groups reads by locus, defines a window enclosed between two custom border sequences, and retains the entire corresponding DNA sequence as haplotype. Haplotype-window is, among many applications, especially useful for high-throughput CRISPR/Cas mutation screens.
Either way, SMAP creates a single integrated haplotype call table across all loci and samples. SMAP haplotyping is extremely versatile and can be applied to highly multiplex amplicon sequencing (HiPlex), Shotgun (e. g. whole genome shotgun (WGS) sequencing, probe capture and RNA-Seq), or Genotyping-by-Sequencing (GBS) data; and to Illumina short reads, PacBio and MinION long reads. SMAP creates discrete genotype calls for individuals of any ploidy or quantitative haplotype frequency spectra for Pool-Seq data, and can scale from tens to thousands of loci and/or samples. SMAP, including the source code written in Python is available at https://gitlab. com.
1  and Supplementary Table  S1 ). In brief, read data may be prepared for read mapping by our new tool GBprocesS (including demultiplexing, trimming, merging forward and reverse reads, quality filtering, etc. ; see https://gbprocess. readthedocs. io/), and reads are mapped onto a reference sequence with any existing read mapper such as BWA-MEM  (Li, 2013) , Bowtie2  (Langmead et al. , 2009) , or Minimap2  (Li, 2018). Then, one of two SMAP modules may be invoked for the actual haplotyping. First, module SMAP haplotype-sites phases a-priori known 'sites' (defined as a series of polymorphic, single-nucleotide-resolution coordinate positions in the reference sequence), bundled at loci that are delineated depending on the type of sequencing library; i. e. based on primer positions for HiPlex data, sliding frames in Shotgun data, or, specifically for GBS data, Stack Mapping Anchor Points as identified with an additional tool called SMAP delineate (Fig.
perenne individual; g) SNP haplotyping in the PotatoMASH HiPlex panel in a tetraploid S. tuberosum individual; h) SNP/SMAP haplotyping in PstI Pool-Seq GBS of a pool of 48 diploid L. perenne individuals. The second module, called SMAP haplotype-window, delineates a locus by defining a so-called "window" enclosed by two locus-specific "border" sequences (coordinates are provided as a custom GFF file). SMAP haplotype-window then identifies the IDs of all reads mapped per locus in a BAM file; retrieves the corresponding sequences from the original FASTQ file (circumventing potential loss of sequence length due to soft-clipping or hard-clipping during read mapping); trims off the two locus-specific border sequences, and retains the entire remaining internal DNA sequence as a unique haplotype (Fig.
2 ). This concept for haplotyping is independent of a-priori variant calling, and is especially useful to de novo discover complex haplotypes derived from combined SNP and indel structures (e. g. derived from CRISPR/Cas genome editing, see De Bruyn et al. , 2020). Finally, in both modules, all unique haplotype counts are listed per locus per sample in an integrated haplotype call table. Both modules apply the same set of functionalities for haplotype frequency filtering, discrete genotype calling and locus and sample quality controls and create tabular and graphically represented summary statistics. The SMAP haplotype call table contains tabular formatted multi-allelic haplotype frequency spectra for Pool-Seq sample sets, or discrete multi-allelic haplotype calls for individual samples (diploids or polyploids), and scales from tens to thousands of loci and/or samples.
GBS is a popular method to sequence a representative fraction (typical range 0. 1% -1%) of a genome to generate genome-wide marker sets. GBS libraries are generated by ligation of sequencing adaptors to restriction enzyme overhangs, followed by size-selective PCR-amplification and sequencing  (Elshire et al. , 2011) , i. e. the NGS equivalent of Amplified Fragment Length Polymorphism (AFLP). SNP calling algorithms typically rely on the comparison of 'piles' of aligned nucleotides to the respective reference nucleotide (Supplementary Fig. S1 ). So, constructing a complete genotype call table requires that a given nucleotide of the reference sequence is consistently covered by a minimal number of reads per sample across the sample set, and any factor that leads to missing read alignment creates missing data points in the genotype call table, thus lowering the power of GBS studies.
We implemented SMAP delineate to systematically investigate GBS read mapping profiles prior to any variant calling (Fig. 3a ), and disentangled the underlying technical and biological factors that cause variation in read mapping profiles. For instance, SMAP delineate saturation curves (Fig. 3c ), reveal if the same number of loci are detected per sample (influenced by technical factors including the choice of enzymes and minimal required total library size (Fig. 3b ); important for genetic studies that rely on linkage disequilibrium), while locus completeness plots reveal whether the same loci are detected across the sample set (Fig. 3d ,e; e. g. influenced by biological factors such as the frequency of SNPs affecting restriction sites, Supplementary Table  S2 ). SMAP delineate thus serves to routinely analyze read mapping distributions across multi-sample GBS datasets (Fig.
General conclusions and perspectives The most prominent advantage of the SMAP package is that the haplotype table comprises a unified notation for multi-allelic variants that capture SNPs, SMAPs, SV junctions, and any combination thereof, so that all sequence variants per locus can be used together in a single integrated quantitative genetic analysis. We have highlighted the relative strengths and complementarity of novel concepts for haplotyping implemented in SMAP haplotype-sites and haplotype-window, and illustrated how various modules work together to extract the most informative haplotype diversity per sample set, depending on NGS library method, sequencing technology, prior knowledge of variants, and type of sample. For instance, initially conceived to explore the underlying causes of read mapping polymorphisms inherent to GBS, SMAP delineate now serves to routinely analyze read mapping distributions across multi-sample GBS datasets.
Module SMAP compare shows the overlap in read mapping profiles across two sample sets (both analyzed by SMAP delineate), e. g. to optimize SMAP delineate parameters, to estimate reproducibility of GBS library preparation across laboratories, to compare GBS Pool-Seq to their constituent individuals, or to evaluate the number of common polymorphisms across genepools or breeding populations. The SMAP utility tools, in combination with complementary concepts to delineate loci such as sliding frames, allow to exploit many other types of NGS libraries and sequence polymorphisms, next to the most commonly used SNPs. Furthermore, SMAP delineate combined with haplotype-sites resolves the genotype calling paradox in GBS data by: 1) recognizing that read mapping polymorphisms exist; 2) systematically positioning read mapping polymorphisms in a coordinate system of SMAPs that mark locus start and end in a read data-driven fashion; and 3) combining SMAPs as a novel type of molecular marker with SNPs to create multi-allelic markers at the haplotype level.
g. Fig. 1a-c , Shotgun panel right locus); or 2) to quantify the relative expression level per haplotype based on sliding frames with neighboring SNPs to evaluate allele-specific expression bias using CDS transcripts as reference and avoiding gapped alignments. While simple haplotype frequency intervals worked surprisingly well to transform haplotype frequency spectra into discrete haplotype calls, we further see potential for using probabilistic models for inherent ploidy assignment and discrete genotype calling. Dominant and dosage calls are currently implemented, but all samples processed together need to be of the same ploidy level. To further develop the SMAP package, we are expanding the set of SMAP utility tools (Supplementary Fig. S6 ), to further support experimental design and downstream genetic analysis (e.
g. a module called SMAP design that designs primers for HiPlex sequencing optionally combined with gRNAs for multiplex CRISPR/Cas genome editing). To facilitate downstream genetic analyses, SMAP haplotype-sites and haplotype-window optionally create the discrete haplotype call table in the format required for Cervus  (Kalinowski et al. , 2007) , a program for parental analysis using multi-allelic genetic markers. Finally, we are developing a module called SMAP effect-prediction to exploit the benefits of sequence-based haplotypes generated by SMAP haplotype-window by reconstructing the genic context of the detected allelic sequence variants, translating the corresponding protein sequence, and quantitatively scoring the sequence similarity to the reference protein as proxy for the impact of an observed mutation on the protein functionality, in the frame of high-throughput screens for naturally occurring and/or CRISPR/Cas-induced mutations.
Acknowledgments We gratefully acknowledge  Jens Renders ,  Rudolf Aelbrecht , and other students of the  Ghent University 2019 Computational Biology  class led by F. V. and  P. D. for optimization of various parts of the initial code. We thank  Hilde Muylle  and  Isabel Roldan-Ruiz  for helpful discussions,  Michaël Kelchtermans  for his contributions to the online manual, and  Marie Pégard  for analyzing alfalfa data. This work was supported by the  European Union 's  Horizon 2020 research and innovation program  ( Marie Sklodowska-Curie Grant  number  797162 ); the  Irish Department of Agriculture, Food and the Marine (DAFM)  through the  Virtual Irish Centre for Crop Improvement (VICCI)  (Project number  14/S/819 ); the  Agency for the promotion of Innovation by Science and Technology (IWT) Flanders  (project  LO-080510 ); the  Research Foundation Flanders  ( FWO  Grant  K208710N ); the project  GrassLandscape  awarded by the  2014   FACCE-JPI ERA-NET+ call Climate Smart Agriculture  (Grant Number:  609398 ) and by the  EC  (Grant number  618105 ), by the  Agence Nationale de la Recherche (ANR)  and the  Institut National de la Recherche Agronomique (INRA -métaprogramme ACCAF) in France , the  Biotechnology and Biological Sciences Research Council (BBSRC)  in the  United-Kingdom , the  Bundesantalt für Landwirtschaft und Ernährung (BLE) in Germany ; the  European Union 's  Horizon 2020 Program for Research & Innovation  under grant agreement no.
727312  (project: " EUCLEG -Breeding forage and grain legumes to increase EU's and China's protein self-sufficiency ") and from the  Ministry of Science and Technology of China  ( 2017YFE0111000 ). The computational resources ( Stevin Supercomputer Infrastructure ) and services used for genotype calling and algorithm optimization were provided by the  VSC (Flemish Supercomputer Center) , funded by  Ghent University in Belgium ,  FWO  and the  Flemish Government -department EWI. E. V. was supported by an  ILVO PhD  fellowship. A. L was supported by a joint  PhD fellowship  at  ILVO  and  VIB (Vlaams Instituut voor Biotechnologie). Y. B. was funded by  Research Foundation Flanders  ( FWO  Grant  G056517N ).
Code availability SMAP is available at https://gitlab. com/truttink/smap/ under the GNU Affero General Public License v3. 0, and a detailed user manual is available at https://ngs-smap. readthedocs. io. Additional tools for downstream analysis of SMAP haplotype tables are available at https://gitlab. com/ybawin/smapapps and https://gitlab. com/ybawin/primerdesign-gbs.
Data availability SRA accession numbers of sequencing data that has been deposited in NCBI, are listed per study in the Online Supplementary Methods. Author contributions E. V. and T. R. conceived and designed the core of the SMAP code. D. S. designed and implemented GBprocesS and SMAP in all its current forms. Code optimization was performed by D. S. , F. V and P. D. Test data analysis was performed by T. R. , S. V. G. , A. H. , Y. B. , J. L. , P. B. , S. Blugeon. M. L. and S. Byrne. T. R. wrote the manuscript with input from all authors. T. R. supervised development of the tool and guided the project to completion. All authors read and approved the final manuscript.
ONLINE CONTENT Any methods, additional references, source data, extended data, supplementary information, acknowledgments, details of author contributions and competing interests and statements of data and code availability, as well as a user manual with description of working procedure, guidelines for accurate read preprocessing and read mapping, command line options, use of arguments, and guidelines for troubleshooting, are available in the Online Supplementary Material and the online manual https://ngs-smap. readthedocs. io/. The Online Supplementary Materials provide the following parts:   Table  S1  | Non-exhaustive overview of various tools for haplotyping. Table  S2  | Abundance of SNPs affecting restriction sites in L. perenne. Table  S3  | Selection of 269 WGS datasets of Oryza sativa for haplotype calling of structural variants.
Ribosome dependence and cleavage preference by TAC toxin in vitro. The nEMOTE data highlighted several potential substrates of HigB TAC , including the frequently cleaved cspA and groES transcripts, whose cleavage sites are conserved in both M. smegmatis and M. tuberculosis (Fig. 2a  and Supplementary Fig. 2a ). In agreement with the in vivo data, we found that both cspA and groES of M. tuberculosis were also bona fide HigB TAC substrates in vitro, as judged by HigB TAC ability to inhibit M. tuberculosis CspA (Fig. 3a ) and GroES (Supplementary Fig. 3a ) synthesis by E. coli ribosomes. The cspA mRNA, which has only one HigB TAC cleavage site in its sequence, i. e. , a CCA codon at the amino acid position 2, was selected as a model substrate transcript to further study HigB TAC mode of action and motif preference in vitro.
4 ). This is in agreement with the nEMOTE data that identified HigB TAC cleavages along different transcripts (Supplementary Fig. 2a ). The in vivo nEMOTE data strongly suggest that HigB TAC cleaves between the second and the third nucleotide within codons, HigB TAC Independent cleavage events with a preference for adenosine at the third position and with CCA codons being the most represented targets (Supplementary Fig. 2b ). We next engineered mutations at each position of the CCA codon of cspA and tested them for translation inhibition and cleavage by HigB TAC in vitro (Fig. 3c, d  and Supplementary Fig. 3d, e ). Strikingly, our data show that the adenosine at the third position is indeed critical for cleavage by HigB TAC , as all of the mutations at this position efficiently prevented both inhibition of protein synthesis and cleavage by the toxin.
CCA is a relatively rare codon in M. smegmatis (0. 3%) and M. tuberculosis (0. 6%), but is present in 51 and 75% of their coding sequences, respectively. Although less represented, codons ACA, UCA, UUA or CUA (coding for threonine, serine, leucine, and leucine, respectively) are also well recognized and cleaved by the toxin in vivo and/or in vitro. Overall, HigB TAC appears to have a strong preference for adenosine at the third position, as found for P. vulgaris HigB  52, 55  , with the first and second position also contributing to specificity, although less significantly. Therefore, HigB TAC cleaves irrespectively of the codon family and has the ability to target a large set of mRNAs as long as they are being translated. Despite a robust identification of the preferred motifs, nEMOTE only identified a few annotated mRNA transcripts that are cleaved following HigB TAC expression (18 in total).
Four of them encode essential proteins in M. smegmatis (seven in M. tuberculosis), suggesting that cleavage of such mRNA targets may contribute to the growth inhibition or cell death induced by this toxin. Although the relatively small number of mRNA targets identified could reflect that only a few types of RNA molecules are cleaved, it is more likely that only a fraction of each RNA species is cleaved. In this case, nEMOTE would consistently detect frequent cleavages in abundant RNAs, while cleavages of low-abundance RNAs (and rare cleavages of abundant RNAs) would only be detected in some RNA samples but at the level of background noise in others  44. An estimation of transcripts abundance in M. smegmatis indicates that all the HigB TAC mRNA targets identified in this work have an average, or above average, abundance in vivo when compared to all the annotated transcripts, suggesting that indeed only well-represented transcripts could be identified with confidence (Supplementary Fig.