dataesr/scientific-paragraphs-categorization huggingface dataset:
5% to 35. 6%). Moreover, we used less permissive Gblocks parameters, reducing positions from 21 152 to 18 463  (Philippe et al. 2011b). Despite the use of the same genes, the same species and the same evolutionary model, trees inferred from the original and revised datasets differed by a Robinson-Foulds distance of 36, i. e. , 18 clades among 74 were different. The distance was increased if a better fitting model (the CAT model instead of the WAG model) was used: 24 bipartitions among 74 (i. e. , a Robinson-Foulds distance of 48) were different between trees inferred based on original  (Dunn et al. 2008 ) and revised  (Philippe et al. 2011b ) datasets. For instance, acoels moved from within Lophotrochozoa to sister of Deuterostomia, Ctenophora moved from sister to all the remaining animals to sister to all but Porifera, or Myzostoma moved from sister of Gnathostomulida + Acoela to within Annelida, all these changes being of great biological significance.
Data quality control No standard for the quality control of a phylogenomic supermatrix has been established yet. Instead, only preliminary attempts for reporting data have been made  (Leebens-Mack et al. 2006). Obviously, manual and semi-automated controls that are in use for single-gene phylogenetics can still be applied, though they are very time-consuming (e. g. , several weeks of tedious work in the case of our two previous studies,  Philippe et al. 2011b and Laurin-Lemay et al. 2012)  and subjective (i. e. , not easily reproducible nor easily assessable by referees). It is therefore of prime importance to develop bioinformatics tools to assemble high-quality datasets and necessary that the phylogenetics community establishes a standard for the quality of a supermatrix for phylogenetic analysis.
Note that errors can be very complex to track down, and even to define. For example, a gene horizontally transferred from a parasite to its host could easily (but erroneously) be viewed as a contaminant, and thus removed from the dataset; yet this would be the right thing to do only when the inference method assumes orthology, since such a xenologous gene might also provide valuable information  (Huang & Gogarten 2006). Similarly, the known case of a frameshift in the MurF gene of Buchnera, which is rescued by polymerase infidelity  (Tamas et al. 2008) , looks very much like a sequencing error in the genomic sequence. While these exceptions certainly do exist, they are rare and can likely be safely ignored in the context of supermatrix construction. Obvious ways of reducing data errors are to improve the sequencing technology, the genome assembly method, the gene annotation protocol and the identification of orthologous genes.
Reviewing the literature,  Morrison (2009)  reported that manual curation is a common practice, with 78% of sequence alignment procedures described in 1280 papers published in 2007 involving human intervention. The landmark work on automatic alignment filtering is the famous Gblocks program  (Castresana 2000) , which (although based on a basic strategy) allows removal of the less reliable parts of an alignment and hence improving phylogenetic inference  (Talavera & Castresana 2007). Gblocks first considers each site independently; depending on the presence of gaps and on the percentage of identical residues, sites are partitioned in the three following categories: "non conserved", "conserved" or "highly conserved". Then reliable blocks of sites are defined by taking into account the degree of conservation of sites inside and around those blocks.
The fact that Gblocks considers the neighbourhood of a site, when deciding whether to keep it or not, is a major reason for its efficiency. By default, the original Gblocks program removed all sites containing gaps. This is generally too strict, especially when dealing with numerous sequences, and a threshold on the percentage of allowed gaps is preferable. Similarly, the percentage of identical residues is a crude measure of site conservation. More recent alignment filtering methods, such as trimAl  (Capella-Gutierrez et al. 2009)  or BMGE  (Criscuolo & Gribaldo 2010) , refine this measure of site conservation through the use of, e. g. , PAM or BLOSUM matrices  (Dayhoff et al. 1978; Henikoff & Henikoff 1992). Moreover, unlike Gblocks, these two filtering software programs do not require the user to provide fixed threshold values for determining sites to be trimmed.
This is particularly useful to identify rogue sequences, e. g. , resulting from contaminations or erroneous homology predictions, before any further analysis. This can even be extended, as done for instance in BMGE, GUIDANCE and FSA, to mask portions of sequences, which is particularly useful to detect annotation errors or frameshifts in coding sequences. The method StatSigMA  (Prakash & Tompa 2005 ) uses an extension of the BLAST statistics  (Altschul & Lipman 1990 ) to detect such nonorthologous portions of sequences  (Prakash & Tompa 2005). Alternatively,  Philippe et al. (unpublished)  introduced HMMCleaner, a program that tackles the same problem by first building a Hidden Markov Model profile of the alignment, and then measuring the score of the different sequence regions along this profile.
This strategy is implemented in transAlign (Bininda-Emonds 2005), TranslatorX  (Abascal et al. 2010) , and Seaview 4  (Gouy et al. 2010) , for example. The main limitation of this approach is the inability to handle unexpected frameshifting indels (either real or due to sequencing errors) that result in (partially) incorrect translations. Affected sequences hence look like highly divergent protein sequences and, in the best case, are removed by alignment filtering methods. The MACSE program handles such situations by simultaneously considering the nucleotide sequences and their amino-acid translations and by explicitly taking into account frameshift events during the alignment process  (Ranwez et al. 2011). This provides an automatic solution to detect undocumented frameshifts in public database sequences and highthroughput sequencing contigs.
Reducing orthology prediction errors Although errors due to the inclusion of non-orthologous sequences in phylogenomics can have drastic consequences on the final results  (Philippe et al. 2011b; Laurin-Lemay et al. 2012) , a hunt for such dubious sequences is rarely performed in phylogenomics and, when it is, fastidious manual approaches are most often used. An obvious type of non-orthology error that can be detected and corrected for is biological (or in silico) sequence contamination. BLAST similarity searches provide an easy way to detect contaminant sequences, as long as the reference database is taxonomically rich. Hence, a much better hit on a sequence from a different clade than on a sequence from the expected clade (e. g. , ciliate for a mollusc sequence) is an excellent indicator that the query sequence is "fishy".
If a sequence had a better hit against Nemastostella than against Lottia, it was considered as a contaminant and discarded. Though sound in its principle, this approach is expected to be limited due to the use of a single cnidarian proteome. Indeed, using multiple cnidarian species, we detected several cnidarian contaminants in the published dataset using a semiautomatic approach (data not shown). Contamination is not always as simple to detect as in the ideal case of a very distantly related contaminant organism. Further, the BLAST approach has some serious limitations. First, it requires knowledge of the phylogeny, which is often the question that needs to be answered in phylogenomic studies. Second, if contamination occurs between closely related species and/or if taxon sampling is too sparse, the BLAST approach is unlikely to detect contaminants.
Instead of looking for unexpected BLAST hits, it is thus preferable to build and check individual gene trees for unexpected bipartitions (i. e. , aberrant placement of some species). Again, this approach was also used by  Kocot et al. (2011). To this end, they inferred a maximum likelihood tree for each individual gene and "manually evaluated  [. ]  any remaining cnidarian contamination in the neomenioid datasets". These were neomenioid sequences that were forming a clade with Nematostella or were part of a polytomy that included Nematostella. Their protocol was nonetheless focused on a single type of errors and more systematic strategies are certainly needed. One example of such an approach was developed to check the orthology in a supermatrix used to reconstruct deep animal relationships  (Philippe et al.
2009). For each gene tree, the presence of every bipartition with a bootstrap support value ≥ 70% was compared to the bipartition list of the tree obtained by concatenating all the genes (this tree is considered as a good approximation of the true, unknown, species tree). The authors found that the number of conflicting bipartitions was small and that incongruences almost always represented only minor rearrangements in the tree (often nearest neighbour interchanges). Consequently, they concluded that paralogy was not playing a significantly negative role in their analysis. While this approach is reasonable for checking that a dataset is sufficiently free of nonorthology errors, it is seriously limited by stochastic error. Indeed, in single-gene phylogenies, most of the nodes are poorly supported and, if the support threshold is reduced (e.
g. , down to 50%), too many nodes conflict with the tree inferred from the supermatrix, whatever the actual quality of the dataset. Moreover, a manual, time-consuming step is needed to interpret each automatically detected conflict to determine whether it is due to a stochastic (e. g. , partial sequence) or systematic (e. g. , long-branch attraction (LBA)) error, or caused by contamination or undetected hidden paralogy. An automatic approach for detecting outlier sequences in a phylogenomic dataset, named Phylo-MCOA (de  Vienne et al. 2012 ) has been developed. It is based on a multivariate approach known as Multiple CO-inertia Analysis (MCOA)  (Chessel & Hanafi 1996). It takes as input a set of gene trees, converts all these trees into pairwise distance matrices (using either nodal or patristic distances between species) and compares all these matrices simultaneously.
An example of such a matrix for a hypothetical dataset of 40 genes and 40 species is presented in Fig. 3. Phylo-MCOA distinguishes two types of outliers, the "complete" outliers and the "cell-by-cell" outliers. Complete outliers (black arrows in Fig. 3 ) represent either species that have a position that is very variable with respect to other species "for all the genes" or genes for which "all the species" have a position that is not concordant with the other species in the other genes. This is clearly the case for two genes (g24 and g32) and three species (t23, t34 and t32) in the example matrix of Fig. 3. Phylo-MCOA provides a simple outlier detection method to identify these complete outliers (see  de Vienne et al. 2012  for details). Cell-by-cell outliers (dashed circles in Fig. 3 ) are specific species in specific genes whose position is not concordant with their position in the other gene trees.
They can only be identified with confidence in the 2WR-matrix after complete outliers have been removed and the analysis performed anew. The final output of Phylo-MCOA is a list of all complete outlier species, all complete outlier genes and all cell-by-cell outliers. These detected outliers may represent either problematic sequences due to contamination, frameshift events, incorrect annotations, erroneous chimerical sequences, wrong orthology assessment, or indicate interesting biological processes such as horizontal gene transfer, gene conversion, incomplete lineage sorting or hybridization. When the goal of the analysis is, as discussed here, to ensure having a dataset containing only orthologous sequences, one should focus on these detected outlier sequences to remove them. As an example, Phylo-MCOA was used on the phylogenomic dataset of  Schierwater et al.
(2009)  known to contain problematic sequences that were manually identified by  Philippe et al. (2011b). By comparing the tree obtained after removing all the outliers identified by Phylo-MCOA and the original tree of  Schierwater et al. (2009) , it appeared that (1) the strong bootstrap support for the monophyly of diploblasts was reduced (from 100% to 63%), as expected from the fact that it is certainly not a monophyletic group, and (2) the monophyly of Porifera disappeared (from 100% in the original tree), in agreement with the very low bootstrap support (36%) obtained by  Philippe et al. (2011b)  on the same dataset after replacing the incorrect sequences they detected by better ones. Even though the automatic approach performed with Phylo-MCOA and the manual approach conducted by  Philippe et al.
Evaluating the impact of missing data Assembling a large and complete dataset with hundreds of orthologous genes and tens to hundreds of species remains a challenge, since a gene may have not yet been sequenced, might have been lost in a lineage, or could have been replaced by a xenologous copy through lateral gene transfer. As a result, many supermatrices were published with a high amount of missing data (e. g. , 55%, 74%, 79% and 81% for  Dunn et al. (2008) ,  Kocot et al. (2011 ), Smith et al. (2011 ) and Hejnol et al. (2009) , respectively). The impact of missing data on the accuracy of phylogenomic inference has received a lot of attention, but studies focused on this issue have reached two opposite conclusions: (1) the effect of the incompleteness is deleterious and sparse supermatrices must be avoided  (Huelsenbeck 1991; Lemmon et al.
2009)  and Fig. 3. Typical output of Phylo-MCOA. A matrix containing as many rows as the number of species and as many columns as the number of genes was computed, in which complete (black arrows) and cell-bycell (dashed circles) outliers can easily be detected. Cells with a high value (dark grey) represent species whose position in a given gene is not concordant with their position in all the other genes. It is thus a measure of distance to the common signal present in the data. (2) missing data are not problematic per se as long as a sufficient number of characters are known for each taxon  (Wiens 2003; Philippe et al. 2004; Wiens & Morrill 2011). Because the impact of missing data has mainly been studied on medium-sized datasets and/or on simulated sequences, these conclusions are not easily transposable to phylogenomic datasets.
To test whether missing data can generate such misleading effects in phylogenomics,  Roure et al. (2013)  assembled an almost complete supermatrix (39 animals, 126 genes for 29 715 amino acids) and progressively increased the proportion of missing genes (20% to 80%) in some species (8 or 27), by introducing missing data following the patchy patterns observed in real transcriptomic-based datasets while controlling for information loss and decrease in taxon sampling. This empirical study demonstrated that a patchy distribution of missing data has an effect similar to the complete removal of species, but is much more detrimental than the complete removal of genes. Missing data indeed exacerbate systematic error leading to LBA artefacts by reducing the ability to detect multiple substitutions, as evidenced by tree length decreasing when the fraction of patchy missing data increases  (Roure et al.
2013). In this respect, it is worth noting that algorithmic approaches have been proposed to extract a subset of the original dataset (by removing genes and/or species) in order to reduce the proportion of missing data while preserving as much information as possible  (Sanderson et al. 2003; Dutheil & Figuet 2015). The concept of "effective number of species", i. e. , the real number of known character states at a typical position, is useful to understand the effect of missing data (Fig. 4 ) and to assist in deciding when to stop adding increasingly incomplete genes to a supermatrix. Though it is fundamentally a property of each column of the alignment, it can be considered globally by computing the product of the number of taxa by the frequency of unambiguous data (e. g. , 94 × 0. 19 = 18 for the  Hejnol et al.
(2009) dataset). A low effective number of species provides an intuitive explanation for the exacerbation of systematic error Fig. 4. Distribution of missing data in phylogenomic datasets. The raw dimensions of a supermatrix (in numbers of genes and taxa) should always be accompanied by an estimate of its level (and ideally distribution) of missing data (A). Indeed, some datasets were advertised as very large when published but required a lot of useless computing power due to a large proportion of missing data (B). More importantly, such datasets are more exposed to data errors, systematic error and phylogenetic artefacts, as their effective number of species is actually quite low for the largest part of their width. In other cases, a targeted completion (in the outgroups) was enough to improve the phylogenetic accuracy (D-E).
In contrast, datasets assembled with the optimal phylogenomic zone in mind combine a sufficient number of genes and taxa while featuring a low proportion of missing data (C). These are the most appropriate datasets to produce accurate phylogenetic relationships. by missing data in a given dataset. The answer to the question "should we add a highly incomplete gene to a phylogenomic supermatrix?" is therefore rather no. It may be better to use a narrower, but almost complete dataset, than a wider and very incomplete one. In contrast, the answer to the question "should we add a very incomplete species to a phylogenomic supermatrix?" is more complex. In agreement with previous studies  (Philippe et al. 2005b; Wiens 2005) , we have shown that including incomplete species, especially slow-evolving ones, helped us to reduce the effects of systematic error  (Roure et al.
2013; Hosner et al. 2016) , especially when the latter is limited (between 20% and 40%). First, the alternative methods that have been suggested as being less sensitive to missing data, such as super-tree, maximum parsimony and gene-partitioning models, turned out to be more sensitive to missing data (albeit not significantly for the latter). Second, the choice of the model of sequence evolution (e. g. , the site-homogeneous WAG + F + Γ 4 model versus the site-heterogeneous CAT + Γ 4 mixture model) had a much larger impact than the level of missing data. In this particular case, the tree inferred from a dataset with 27 species having 80% of missing data using CAT + Γ 4 was closer to the best phylogenetic estimate than the tree inferred from the complete dataset using WAG + F + Γ 4 , especially for the position of the fast-evolving taxa  (Roure et al.
Limiting systematic error Once the issue of data errors in the supermatrix assembly is overcome, systematic error is the main, if not the sole, limitation to the power of phylogenomics. Felsenstein (1978)  described the LBA artefact almost 40 years ago, demonstrating that maximum parsimony could converge towards an incorrect solution with additional characters. Although the number of characters used in phylogenomics (typically from tens of thousands up to a few million) is far from infinite (i. e. , the convergence of a method), the issue of statistical inconsistency is important and should not be neglected. The approaches used to eschew inconsistency have been reviewed several times (e. g. ,  Olsen 1987; Felsenstein 1988; Sanderson & Shaffer 2002; Soltis et al. 2004; Philippe et al. 2005a Philippe et al.
(2011)  were able to overcome this composition-based artefact and to recover a tree identical to the one obtained from amino-acid sequences. Finally, recent studies of genome datasets in mammals  (Romiguier et al. 2013 ) and birds  (Jarvis et al. 2014)  have revealed that AT-rich genes hold a much clearer phylogenetic signal than GC-rich genes, which are subjects to GC-biased geneconversion (Eyre-Walker 1993) episodes possibly leading to an increased heterogeneity in both base composition and evolutionary rate among taxa. Developing methods to detect and remove the sites and the species that violate the model of sequence evolution most significantly is a promising avenue of research. Posterior predictive tests as implemented in programs such as PhyloBayes 3  (Lartillot et al. 2013)  and PHYCAS  (Lewis et al.
2015 ) allow evaluation of model fit to the data or part of the data. Sun et al. (2016)  have recently used this kind of approach to identify sites in a whole chloroplast dataset that did not fit the model of sequence evolution. They showed that removing those sites from the analysis significantly improved phylogenetic inference of green algae relationships. Even if data removal is intellectually less satisfying than developing a "perfect" model of evolution fully accounting for the complexity of the data, this kind of approach could be similarly efficient and certainly requires less computational resources, since these reduced datasets can be analysed with simpler models.
Conclusions With the democratization of high-throughput sequencing, acquiring new genome data has ceased to be the limiting factor in phylogenomics, except maybe for organisms that are difficult to sample from the environment. In contrast, phylogeneticists are now faced with a rise in data errors, which stems from a flood of increasingly bogus genomic data that has become intractable by hand. However, with more effort from the community, notably by investing thinking and time in the development of adequate bioinformatics tools, the issue of data errors should be solved in the next few years. Given the size of current and upcoming phylogenomic datasets (thousands of genes for hundreds -and soon thousands -of species), computational requirements are emerging as the new limiting factor. As multiple possible improvements are theoretically able to limit systematic error (and artefacts) during phylogenetic inference, we will need both simulation and empirical studies to design the best compromise between using many species with a simple model and using few species with a complex model.
Introduction With the advent of Next Generation Sequencing (NGS) techniques, sequencing genomes has become routine. While this is of tremendous interest by providing a profusion of sequence data, contributing accurate knowledge coupled to the sequences has become a nightmare  (Zallot et al. , 2016). The main reason for this dire state of affairs is that automated in silico pipelines draw 'knowledge' by inference, relying primarily on protein sequence similarity analysis, with the function tag extracted from databases that basically lack experimental information. Worse, this approach most often uses the majority rule (a function is deemed correct if it is found in the majority of annotations). Several correction systems have been devised to improve this highly inadequate approach, but the fact is that in the absence of a process allowing experimental validation (direct or indirect) of annotations, errors continue to percolate through the system  (Gilks et al.
This was expected because the sequence was obtained in different laboratories, where a variety of experimental protocols was used. It is also likely that mutations occurred even during the cultivation steps that are a prerequisite to sequencing. The genome was therefore entirely resequenced using NGS methods ten years later  (Barbe et al. , 2009). It can now be expected that, barring the inevitable mutations that appear during propagation in laboratories [see the situation for E. coli  (Soupene et al. , 2003) ], this final sequence corresponds to an exact sequence [International Nucleotide Sequence Database Collaboration (INSDC) AccNum AL009126. 2], that does not need to be re-sequenced. In contrast, sequence annotations inevitably keep changing as the identification of gene function improves almost on a daily basis.