dataesr/scientific-paragraphs-categorization huggingface dataset:
The genes only recovered with the zebrafish as a guide species probably mostly correspond to fish-specific gene families. To interpret the differences between the pipelines in terms of their power to detect transcripts rather than in terms of whether they can detect fishspecific gene families or not, we discuss in the following the assembly with the Human as guide for both the mouse and the stickleback and for both methods. To examine the completeness of both assemblies, we associated by RBH the target species reference transcripts to the reconstructed sequences. Out of all 22 060 mouse reference sequences, 10 672 sequences were found by both assemblies, 828 sequences only by CAARS, 77 only by the standard assembly (Fig. 2A ). Therefore, in this case, CAARS retrieves 7% more transcripts than a pipeline classically used for de novo assemblies.
In the mouse, 93% of the reference sequences have an associated sequence in the CAARS assembly and 89% in the standard assembly (Fig. 2A ). In the stickleback, the improvement of CAARS over the standard assembly is more pronounced: 66% compared to 59% (Fig. 2B ). As CAARS annotates transcripts using a phylogeny, it is expected to be more effective than the standard pipeline, which used a RBH annotation, to retrieve genes in the target species that have been duplicated since the divergence with the guide species (one-tomany or many-to-many orthologs in Ensembl Compara). Such orthologs correspond to 8% of the expressed genes (>1 count/pb) in the mouse and 30% in the stickleback. Expectedly, they are more often retrieved by CAARS than the standard pipeline, whether for the mouse, where 63. 0% of them are retrieved compared to 57.
CAARS assembly provides more complete transcripts The completeness of an assembly cannot be assessed solely on the basis of the number of recovered transcripts. We measured the coverage of each reference transcript, and compared the results between the two assemblies for each target species. Gene coverage was estimated on pairs of sequences with a transcript from the reference transcriptome and the corresponding reconstructed transcript. We computed the percentage of the reference transcript that aligns with the reconstructed transcript (P reference ) for 11 577 Ensembl CDS with a matched sequence in at least one of the assemblies (Fig. 2C ). Both for the mouse and the stickleback, CAARS better recovers the reference transcripts. In the experiment on mouse libraries, the percentage of reconstruction of reference transcripts is identical for 8937 sequences, better in the CAARS assembly for 2509 sequences (with an average increase of 24.
CAARS produces sets of orthologs defined by phylogeny CAARS returns MSA and reconciled gene trees ready to use for comparative analyses. From these reconciled gene trees, the user may use CAARS to infer orthology relationships between all sequences. This information is stored in a table which can be easily mined to retrieve all one-to-one orthologs for a given subset of species (Fig. 3 ). These subsets can include, or not, the target species. For example, we find 4850 sets of one-to-one orthologs with one gene per mammalian species of our dataset (Fig. 3 ). This is substantially more than the number found by the equivalent request in Ensembl Compara (4505 sets of genes with high confidence one-to-one orthology relationship), a gain attributable to our reconciliation step, which improves gene trees  (Boussau et al.
, 2013). We also extracted sets of orthologs for subset of species that include a target species, and obtained reasonable numbers (8705 for Human/mouse comparisons, 6435 for comparisons across rodents, 5 666 for comparisons across fishes Fig. 3 ). We cannot compare these numbers to numbers from the Compara database, because the mouse or stickleback gene complements are partial, being reconstructed from libraries of specific organs.
Expression levels are at least as well estimated by CAARS as with the standard pipeline. The improvement over the standard pipeline can be attributed to two novel features: (i) CAARS implements a trans-species assembly, based on one or several guide sequences, which can be distantly related. This is demonstrated here with mouse and stickleback RNA-Seq data assembled using a very distant Human guide species; (ii) CAARS annotates the transcripts by integrating them in phylogenies built with a set of helper sequences. This is demonstrated here with families from the Ensembl Compara database. In addition to the steps of transcript detection and assembly, CAARS generates gene alignments, trees and sets of orthologs that can be directly used in subsequent comparative analyses. Notably, we found that it could recover more sets of orthologs than the Ensembl Compara pipeline and was better at recovering one-to-many or many-to-many orthologs than the standard pipeline, probably because it relies on gene trees reconstructed with a reconciliation approach.
, 2016; Torres-Oliva et al. , 2016; Ungaro et al. , 2017; Vijay et al. , 2013). For instance,  Vijay et al. (2013)  recommends not to map directly on the guide genome when there is more than 15% of sequence divergence between the target and the guide species, which corresponds to the median nucleotide divergence in one-to-one orthologs between mouse and Human  (Church et al. , 2009). The phylogenetic framework used by CAARS allows identifying redundant transcripts that have been assembled more than once, and collapsing them or selecting the longest (the default). Gene trees are also used by CAARS to identify incomplete transcripts by comparison with neighboring sequences and filter them out (Supplementary Fig. S5 ). It remains to be seen how CAARS would behave on datasets containing lots of recent duplicates; in particular, the option to merge monophyletic transcripts from the same species may create chimeric transcripts and should be used knowingly.
A limitation of CAARS remains the usage in input of MSAs. However, nowadays, there are several public database containing such MSAs [Orthomam  (Ranwez et al. , 2007) , Hogenom  (Penel et al. , 2009 ), PhylomeDB (Huerta-Cepas et al. , 2014) , TreeFam  (Finn et al. , 2014) , EnsemblCompara  (Herrero et al. , 2016) ].
CAARS is robust and easy to install and use CAARS is a complex pipeline combining existing software and newly developed programs, and is built to be able to analyze Fig. 3. CAARS outputs ready to use sets of one-to-one orthologs, inferred from reconciled phylogenies. Each column corresponds to the number of sets of one-to-one orthologs containing at least the species indicated by circles. Empty circles correspond to target species (reconstructed with RNA-Seq data), filled circles to guide or helper species (genome available) thousands of gene families at once. In particular, CAARS includes Apytram  (Rey et al. , 2017) , a multi-species and more accurate reimplementation of TRAM  (Johnson et al. , 2013) , which initially introduced the idea of a trans-species assembler at the level of a single gene.
For robustness and traceability, and to enable recovery upon error or iterative use, it uses the bistro library  (Veber, 2017). For an easy installation, we packaged CAARS into a Docker image, so that there is no need to install any dependency once Docker has been installed on the system, which can be a Mac, Windows or Linux system. Further, by using Docker we ensure that the intended versions of all tools of the pipeline will be used, irrespective of what is installed on the host machine system. The results produced by CAARS are thus fully reproducible. Finally, the use of Docker ensures only minimal penalties on computational efficiency. However, users can also opt to install the full pipeline without using Docker. Here we demonstrated the use of CAARS at the whole transcriptome level but the program can be used at a much smaller scale, for a single gene family, or even for a single gene.
CAARS may also be used for integrating transcripts obtained from a pre-existing assembly into gene families. This is easily feasible by switching off the step of assisted reconstruction in the input option file (explained in the tutorial on CAARS's website). Although CAARS is slower than the standard pipeline, it provides not only the assembly and annotation, but also gene family alignments, reconciled genes trees and sets of orthologous genes. In many cases, such data may be used directly for subsequent analyses.
Conclusion We have introduced CAARS, a new pipeline for the comparative assembly and annotation of transcripts in non-model species. Because it operates within a phylogenetic framework, it can use both closely related and distantly related species. In addition to annotated transcripts, it also provides gene family alignments and trees built using state-of-the-art methods, which can be directly used for downstream analyses. On data coming from the Ensembl database, it compared favorably to a pipeline combining Trinity and BLAST, and provided more complete sets of orthologs than Ensembl. CAARS could therefore be used in a variety of situations where transcript assembly needs to be of high quality, for instance for comparing gene expression or gene sequences across species.
e. , for a single dataset, practitioners cannot simply apply the best w learned for classification on clustering or vice versa. In addition, we will demonstrate that the appropriate amount of warping not only depends on the data structure, but also on the dataset size. Thus, even if a practitioner knows the best setting for a given dataset, they will likely be at a lost if they apply that setting on a bigger size version of that data. All these issues seem largely unknown or at least unappreciated in the community. In this work, we demonstrate the importance of setting DTW's warping window width correctly, and we also propose novel methods to learn this parameter in both supervised and unsupervised settings. The algorithms we propose to learn w can produce significant improvements in classification accuracy and clustering quality.
Fig. 1  illustrates this sensitivity to w for clustering under DTW. Fig. 2  and Fig. 3  similarly demonstrate the sensitivity to w for classification. Note that the Euclidean distance is a special case of DTW when the warping constraint w is equal to 0. Fig. 1  Rand-Index vs. warping window (w) width for three datasets, using a density-based clustering algorithm  (Begum et al. 2015). A larger value of w can make things better, worse or have no effect. Fig. 1  shows how changing w affects the quality of clustering on three different datasets. For Two Patterns, increasing the amount of warping steadily improves clustering quality until it reaches perfection with w = 9. In contrast, for Swedish Leaf, a higher w reduces the quality of clustering from a very impressive (for a 15-class problem) Rand-Index of 0.
87 at w = 0 to a stunningly low score of 0. 32 at w = 10. This finding is more surprising given that allowing some warping improves the classification accuracy of this dataset slightly (H. Ding et al. 2008). These results indicate that blindly using the Euclidean distance for clustering (i. e. w = 0) will yield poor results on some datasets. Likewise, another practitioner, perhaps motivated by the observation that DTW generally helps in classification problems  (Bagnall et al. 2017; Bagnall and Lines 2014; H. Ding et al. 2008) , and thus simply clusters with a hard-coded value of w set at 10, will also do poorly on some datasets  (Paparrizos and Gravano 2015). The Coffee dataset is unusual (and empirically rare), since it is virtually unaffected by varying w (in Fig. 1  it is 0. 48 when w = 0 and 0.
49 everywhere else), but even here it is still possible to make a poor decision. The time taken to compute DTW with a w = 0 (denoted hereafter as cDTW 0 ) is about four orders of magnitude less than the time to compute cDTW 100. Thus, unnecessary large values of w incur a huge computational burden that produces no improvement. Train error-rate Error-rate which is a specific case of the k-NN algorithm when k = 1. This is the method used by the authors of the UCR Archive  (Chen et al. 2015)  (and thus, reflected in hundreds of papers, for example  Deng et al. (2013)  and  Górecki and Łuczak (2013) ). Fig. 2  demonstrates that on many datasets, this simple method produces favorable results. It predicts the correct optimal value of w for CinC_ECG, and it is only off slightly for CBF and 50words.
e. the mixture weights being learned by cross-validation). Among the datasets successfully considered are Lightning2 and Lightning7. The authors note that they can reduce the error-rate of Lightning2 to 13. 11%, but a better choice of w for 1-NN-DTW could significantly beat this with just an 8. 2% error-rate. Likewise, with Lightning7, their method can reduce the error-rate to 32. 88%, which is impressive given a high error-rate of 42. 47% with the Euclidean distance. However, if we use w = 8 for this dataset, 1-NN-DTW can achieve a much better error-rate of 23. 28%. In a follow-up paper  (Górecki and Łuczak 2014) , the authors further improve on the previous method by adding the DTW distance between transforms of time series. Now the new method can match the performance provided by DTW with the best w for the two datasets mentioned above.
2012). This work unifies two previous research efforts  (Dau, Begum, and Keogh 2016)  and  (Dau et al. 2017 ) under a coherent theme: learning DTW's warping window width for time series data mining applications. We have (re)structured these texts to tell a single narrative, that carefully setting DTW's warping window width offers more "bang-foryour-buck" than any other simple change you can make. Integrating these two papers in the current work allows us to more forcefully make a point that was only made in passing previously  (Dau, Begum, and Keogh 2016; Dau et al. 2017). In general, on any given dataset, the best warping window width for clustering is not the best warping window width for classification. The rest of this the paper is organized as follows. In Section 2, we explain the background material of the problem we are solving.
Dynamic Time Warping DTW is a distance measure that originated in the speech recognition community. Recent work strongly suggests that DTW is the best distance measure for many data mining problems (H. Ding et al. 2008). In an influential paper  (Rakthanmanon et al. 2012) , authors state, "after an exhaustive literature search of more than 800 papers, we are not aware of any distance measure that has been shown to outperform DTW by a statistically significant amount," and very recent independent work has empirically confirmed this with exhaustive experiments  (Paparrizos and Gravano 2015). Of course, these results must come with several caveats, the most important of which is that almost all papers (including this one) test only on data from the UCR Archive  (Chen et al. 2015). While the archive is large and diverse, it reflects only distribution of datasets the curators could make or obtain, not the distribution of real-world problems that are worthy of addressing.
Nevertheless, it is telling that in a very competitive research area, there are at least two dozen papers published on time series classification each year, there is still no technique that unambiguously beats DTW on more than half the datasets in the archive. As illustrated in Fig. 4. left, DTW allows a one-to-many mapping between data points, thus enabling a meaningful comparison between two time series that have similar shapes but are locally out of phase. To find the warping path W, we construct the distance matrix between the two time series Q and C. Each element (i, j) in this matrix is the squared Euclidean distance between the  ℎ point of Q and  ℎ point of C. The warping path W is a set of contiguous matrix elements that define the alignment between Q and C. The  ℎ element of W is defined as  = (,).
4. right, the warping path cannot visit the gray cells. The constrained DTW helps avoid pathological mappings between two time series when one point in the first time series is mapped to too many points in the other time series. For example, DTW should be able to map a short heartbeat to a longer heartbeat, but it would never make sense to map a single heartbeat to ten heartbeats. In addition, the constraints have the additional benefit of reducing the computation cost by narrowing the search for qualified paths. A typical constraint is the Sakoe-Chiba Band  (Sakoe and Chiba 1978) , which expresses w as a percentage of the time series length. We denote DTW with a constraint of w as cDTW w. The Euclidean distance between the two time series is a special case of DTW when w is set to 0%, enforcing a one-to-one mapping between data points.
To help illustrate this, we will create a synthetic dataset, which we call Single Plateau (SP). This dataset (and all others in this paper) is available at the paper supporting webpage (Supporting Page 2018). Each item in the dataset consists of a vector of 500 random numbers taken from a standard Gaussian. We add a "plateau" of height 100 and with a length randomly chosen in the range five to twenty to each exemplar. If the plateau's location falls in the range of 1 to 250, it is in class A. If it is between 300 and 500, it is in class B. The plateau never appears in the middle of the time series; Fig. 5  shows examples from each class. Note that while the SP dataset is synthetic, it closely models several real datasets, including yearly "snow-melt" time series, collect by the National Snow and Ice Data Center (NSIDC) in Boulder, Colorado and used as a critical resource for scientists studying climate change  (Hu et al.
The intrinsic variability of the time axis If we cluster SP with cDTW 0 , we obtain a "random" clustering as shown in Fig. 6. left. This is not surprising, as this is clearly a dataset that needs a warping-invariant distance measure. If we re-cluster using cDTW 10 , we obtain a clustering that correctly separates the two classes (in Fig. 6. center). Thus far, these observations coincide with most of the community's intuition. However, what happens when we cluster using cDTW 100 ? Again, we obtain a clustering that appears essentially random (Fig. 6. right). This notion that "a little warping is a good thing, but too much warping is a bad thing" is known (although perhaps underappreciated  (Ratanamahatana and Keogh 2005) ) for time series classification  (Chen et al. 2013) ; however, we believe that this is the first explicit demonstration of the effect for clustering (Fig.
The size of the dataset It might also be imagined that we could discover the best warping window width for a given data type and just use that setting for all future datasets from the domain. For example, we might imagine that for the gesture-recognition-for-tall-males dataset, cDTW 5 is generally best, but for the heartbeat-classification-for-the-elderly dataset, cDTW 13 is generally best. However, we can dash such a hope with the following observation: the best value for w also depends on the size of the dataset. To see this, we can classify increasing large instances of the SP dataset. For each size, we search over all possible values of w and record the value that minimizes the error-rate of LOO cross-validation. Fig. 7  shows the result, averaged over 100 runs. Consistent with observations in  (Ratanamahatana and Keogh 2005) , small datasets tend to require much larger settings of w compared to larger ones.
Note that this size versus the best curve for w is different for different datasets. Thus, we cannot generalize the best setting for w on one subset of a dataset to a different sized subset of the same dataset. As shown in Fig. 7 , the best value for w on this dataset, given that it contains 32 objects, is 46. Let us further consider this particular sized subset of the training set. Fig. 8  displays the effect of w on the misclassification rate of the 32-objects SP dataset. We can see that allowing too much warping is almost as detrimental as too little warping. In this case, the w vs. error-rate curve has a broad flat valley, meaning that even if we choose a w value that is too large or small, we could still achieve low misclassification. However, as Fig. 8  suggests, this curve can take on more complex shapes, which makes the choice of w more critical.
The effect of the shapes of the time series A good value for w depends not only on the intrinsic variability of the time axis and the size of the dataset, but it is also dependent upon the time series shapes. We can illustrate this latter point with a simple experiment. We created two near identical datasets, Slim Plateau and Broad Plateau, which, as their names suggest, differ only in the width of the plateaus (see Fig. 9 ). In both datasets, one class has a plateau in the first half, and the other class has a plateau in the second half. As shown in the leftmost column of Fig. 9 , we can see that both variants cluster well under cDTW 0 (i. e. Euclidean distance). What would happen if we added an identical amount of random warping to both datasets and clustered them again using cDTW 0 ? (We will explain how we can add synthetic warping to a time series in Section 4.
1. 8). As we can see in the rightmost column of Fig. 9 , the clustering of Slim Plateau becomes essentially random, whereas Broad Plateau is basically unaffected. The critical message from this experiment is as follows. In this pathological example, we can measure exactly how much warping there is in a dataset because we placed it there. But even in this case, we cannot use the amount of warping added to guide the choice of w. Even with a lot of warping in the time axis, the best value of w could still be as low as zero, depending on the time series shapes and the size of the dataset. In summary, the best value of w depends on both the data size and the structure of the data. This fact bodes ill for any attempt to learn a fixed one-time domain independent value for it. There is not a single prototypical w vs.
Unsupervised Settings In the introduction, we claimed that the best setting of w for classification is generally not an indicator of the best setting of w for clustering. Since this assumption has been explicitly made, but never formally tested multiple times in the literature  (Paparrizos and Gravano 2015) , we will demonstrate that it is unwarranted. In Fig. 10 , we show both the Rand-Index and the accuracy for two datasets. In retrospect, it is unsurprising that these values are weakly related. For 1-NN classification (the most commonly used classification technique in the literature (H. Ding et al. 2008; Ratanamahatana and Keogh 2005) , only the distance between the unlabeled exemplar and its single nearest neighbor matters. However, for clustering, the mutual distance among small groups of objects matters.
2015). For time series clustering, we do not have access to labeled data. Common practices involve using as large a value of w as the computation resources permit, directly applying the w learned from classification of the same problem domain and resorting to a fixed w value that is known to work reasonably well for most tasks. For example, in a recent highly cited recent paper, the authors noted, "…we use as window 5%, for cDTW 5 , and 10%, for cDTW 10 , of the length of the time series of each dataset; this second case is more realistic for an unsupervised setting such as clustering"  (Paparrizos and Gravano 2015). However, as our motivating examples demonstrate, these practices still require compromising the quality of clustering/classification. For many real-world problems, even a small increase in accuracy matters.