dataesr/scientific-paragraphs-categorization huggingface dataset:
Simulated Nanopore reads from the HIV-genome dataset To test the robustness of SHERPAS to high error rates typical of long read sequencing technologies, we also built a dataset of reads generated with NanoSim-H, a simulator of Oxford Nanopore reads  (Yang et al. , 2017; Břinda et al. , 2018). For each query in the HIV-genome dataset, we generated a single simulated read using NanoSim-H with minimum and maximum length set to 1000 and 9000, respectively, and rate of unaligned reads set to 0. All other parameters were left to their default values. A total of 3000 simulated reads, with average length about 5. 9 kbp, were thus obtained. The reference alignment used for this dataset is the same as that for the HIV-genome dataset. (See the Suppl. Materials, Sec. 5. 5 for details. ).
Running the experiments For each of the datasets described in Sections 3. 3 to 3. 6, a reference tree was constructed from the reference alignment with PhyML 3. 3  (Guindon et al. , 2010)  using GTR + + I as substitution model. Alignment and tree were given as inputs to a customized version of RAPPAS that built a pkDB using parameters k = 10 and threshold parameter 1. 5 (called "omega"). We ran SHERPAS with 8 parameters combinations: SHERPASreduced for ◊ R oe {0. 90, 0. 99} and window size in {300, 500}, and SHERPAS-full for ◊ F oe {1, 100} and window size again in {300, 500}. Using two values for each parameter allows us to gauge their impact on the accuracy of SHERPAS. We also ran jpHMM using its default behavior for HIV and HBV, with and without the option -Q blat to speed-up its execution.
See Sec. 3. 1 for motivation regarding the choice of jpHMM for comparisons. For the HIV-pol dataset (Sec. Column "Mbp" reports the total size of the query dataset in Mbp. Column "#br. " reports the number of branches for which the full pkDB (reduced pkDB) stores information. "R" and "F" distinguish between SHERPAS-reduced and SHERPAS-full, respectively. "HBV-g" and "HIV-g" refer to the HBV-genome and HIV-genome datasets, respectively. "HIV-LR" refers to the dataset of simulated long reads. All times are measured in minutes (m) and seconds (s). software  (Kosakovsky Pond et al. , 2009) , so we included them in our comparisons. The commands used for all these operations and links to files used -including the pkDBs constructed by RAPPAS-are reported for reproducibility in the Suppl. Materials (Sec.
Running times Table  1  shows the running times of SHERPAS-full, SHERPAS-reduced and of two ways of executing jpHMM, that is, with and without the -Q blat option to speed-up its execution. We do not include the time necessary to construct the pkDBs with RAPPAS, as we assume that the pkDB has been obtained prior to the analysis. (To this end, SHERPAS is distributed with the 3 pkDBs used in the experiments reported here. ) Moreover, the numerical parameters of SHERPAS (the ◊ thresholds and the window size) have very little impact on its running time. For this reason, we only report runtimes for default parameters. The running times for jpHMM could not be obtained in two cases for the following reasons:  (1)  for the HBV-genome dataset, we must run jpHMM with the -C option for circular queries, which automatically activates the -Q blat option;  (2)  for the simulated Nanopore HIV reads, the -Q blat option resulted in the program failing to execute, probably because of the di culty of aligning error-rich reads.
SHERPAS is orders of magnitude faster than jpHMM. Compared to jpHMM with the -Q blat option, SHERPAS-full is hundreds of times faster, while SHERPAS-reduced is thousands of times faster. Datasets that took days for jpHMM -Q blat to analyse, can be analyzed by SHERPAS in a matter of minutes, or even seconds. The running time of SHERPAS essentially depends on two characteristics of the dataset. First, it scales linearly with the amount of data to analyse (number of queries and their lengths). Second, it is also related to the number of branches for which some information is stored in the pkDB. In the full version, this number is proportional to the size of the reference tree, while in the reduced version it is equal to the number of root branches. These numbers are reported in the first two columns of Table  1.
See the Suppl. Materials (Sec. 2. 6) for a detailed complexity analysis of the algorithms implemented in SHERPAS. Consistent with the expectations above, the speed-up obtained with SHERPAS-reduced relative to SHERPAS-full is related to the strength of the reduction in the number of branches in the pkDB: the speed-up is moderate for HIV-pol (from 332 to 23 branches), but much   9 500  7. 5 89. 8 97. 1 83. 6 8. 5 6. 3 1. 6  SHERPAS R 0. 9 300  8. 8 89. 4 98. 0 81. 9 12. 6 4. 0 1. 5  SHERPAS R 0. 99 500 17. 0 81. 2 97. 9 82. 6 3. 0 13. 1 1. 3  SHERPAS R 0. 99 300 21. 0 78. 2 98. 8 82. 3 3. 6 12. 9 1   2. Accuracies observed on the HIV-pol dataset. jpHMM-Qb stands for jpHMM with the -Q blat (fast) option. "R" and "F" distinguish between SHERPAS-reduced and SHERPAS-full, respectively. Columns "thr" and "w" report the threshold and window-size used by SHERPAS.
Column "N/A" reports the percentage of sites that are not assigned to any strain. Columns "sens" and "prec" report site-wise sensitivity and precision (in percentage), respectively. Columns "m", "sup", "sub" and "mm" report the percentages of mosaic matches, supersets, subsets and mismatches, respectively. (See Sec. 3. 2 for definitions. ) more pronounced for HBV-genome (from 676 to 8 branches), and for the two whole-genome HIV datasets (from 1760 to 20 branches). As for the di erences across di erent datasets, it is not surprising that the dataset that results in the longest running time for SHERPAS is HIV-genome: its set of queries has the largest aggregate size, and the number of branches in the full pkDB is by far the largest. Running times for HIV-LR (the simulated Nanopore reads dataset) are lower than those for HIV-genome because the simulated reads are in general shorter than the whole genome.
HIV-pol dataset Table  2  compares the accuracy of inter-strain recombination detection methods (see Sec. 3. 7) on the HIV-pol dataset. SCUEAL and jpHMM achieve high accuracies overall on this dataset. Here, SCUEAL and jpHMM use di erent reference alignments, and two strains (A and N) present in some of the queries cannot be recognized by jpHMM (see  Sec. 3. 3). We also observed that many of the pre-queries that Kosakovsky  Pond et al. (2009)  used to construct the queries in this dataset are in fact part of the reference alignment for HIV used by jpHMM. For these reasons, it is not a good idea to draw conclusions about the relative performance of SCUEAL and jpHMM here. Overall, the accuracies displayed by SHERPAS on this dataset are not as good as those of the other methods, especially in terms of sitewise sensitivity and mosaic measures.
The low sensitivity is due to the high incidence of unassigned regions, which is particularly pronounced for SHERPAS-reduced and high values of the thresholds. On the other hand, for SHERPAS-full, a high value of the threshold (◊ F = 100) results in a better site-wise precision than SCUEAL and jpHMM, and in mosaic measures that are almost as good as those of SCUEAL and jpHMM (frequency of mosaic matches: 88. 4%-89% vs. 90%-93. 2%). We also observe that on this dataset SHERPAS-full is generally more accurate than SHERPAS-reduced. This is not surprising, as SHERPAS-reduced uses far less pre-computed information (a much smaller pkDB) than SHERPAS-full. As for the e ect of window size, smaller windows consistently result in higher site-wise precision, and lower frequencies of mosaic matches. This appears to be due to the fact that a smaller window "switches" more easily between di erent strains and therefore has a tendency to produce finer classifications, but more fragmented mosaics.
HBV-genome dataset The results in Table  3  show that, again, jpHMM has a very high overall accuracy, which is rarely matched by SHERPAS. Some of the observations made for HIV-pol can be re-iterated here: again, the site-wise sensitivity of SHERPAS is markedly lower than that of jpHMM, and again, as expected, increasing the thresholds deteriorates sensitivity, and improves mosaic accuracy. Interestingly, on this dataset there does not seem to be any consistent di erence between the accuracies of SHERPAS-full and SHERPAS-reduced. This may have something to do with the nature of the reference tree for HBV, where the 8 strains are monophyletic and well-delimited by relatively long root branches. (Which is not the case for all the strains in HIV-1. ) This may imply that for HBV, the phylo-k-mers inferred for the root branches represent well their respective strains.
HIV-genome dataset The results for the HIV-genome dataset, shown in Table  4 , show a slightly di erent pattern from the other datasets. On the one hand, jpHMM and SHERPAS-full have similar site-wise accuracy measures. Unlike in the previous datasets, the sensitivity of SHERPAS-full is higher than that of jpHMM in 3 cases out of 4. On the other hand, the frequency of mosaic matches for SHERPAS is now substantially lower than that of jpHMM. These seemingly contradictory observations can be explained by inspecting the outputs of SHERPAS and jpHMM on the queries in this dataset. The Suppl. Materials (Annex C) contain an illustration of the outputs of SHERPAS-full and jpHMM on the first 100 queries out of the 3000 in this dataset. An important observation is that the partition produced by SHERPAS often includes short erroneous fragments (that is, that were not present in the correct partition of the query).
For example, among the first 10 queries shown in the Suppl. Materials, 5 queries present such short erroneous fragments  (queries 2, 3, 4, 6, 9;  in some cases the erroneous fragment is so short that it is di cult to observe without zooming). The output of SHERPAS in Fig. 1  (corresponding to query 56) is also an example of this phenomenon: note the short erroneous fragment from strain D. A consequence of this behavior of SHERPAS is that, although its output is usually close to the correct partition, the mosaics it i i "output" -2020/12/1 -12:10 -page 7 -#7 i i i i i i   2. produces are often supersets of the correct mosaics. This phenomenon was also present in the other datasets, as can be seen in the frequencies of supersets, which are always higher than in the other methods (see again Tables  2  and 3 ).
However, here this becomes more visible because the queries are about 3 to 5 times longer than in the other datasets, meaning that the probability of observing such erroneous short fragments in one query increases significantly. As we discuss in Sec. 5. 1, when using SHERPAS to screen for recombinants, supersets should be regarded as far less serious errors than subsets or mismatches. From Table  4 , it is easy to check that here the aggregate frequency of subsets and mismatches is higher for jpHMM (about 18%) than in all 4 runs of SHERPAS-full. The lower sensitivity of jpHMM relatively to the other datasets is due to its behavior at the two ends of queries spanning a whole HIV genome. As can be seen in the Suppl. Materials (Annex C), jpHMM often leaves the ends of a HIV-genome query as unassigned (N/A).
This is likely due to the di culty of alignment and of profile-based modeling in those peripheral regions. Like in the HIV-pol dataset, we note that SHERPAS-full tends to be slightly more accurate than SHERPAS-reduced, in terms of sitewise measures. Using a window of size 300 instead of 500, strongly reduces the frequency of mosaic matches, which is again due to a higher frequency of short erroneous fragments. However, it consistently reduces the aggregate frequency of subsets and mismatches (not shown) which may be important for screening purposes (Sec. 5. 1).
Simulated Nanopore reads from the HIV-genome dataset The results in Table  5  show that the simulated Nanopore reads pose a significant challenge to jpHMM and SHERPAS. This is not surprising, given the high error rates that characterize these reads. We refer to  Yang et al. (2017)  and its Suppl. Materials for an in-depth analysis of these error rates. Strikingly, however, jpHMM is much more negatively a ected by the simulated Nanopore sequencing errors than SHERPAS. Note that if a classifier was to choose randomly one of the 14 strains at every site, its sensitivity and precision would be 1 /14 = 7. 1%, but if it was to classify every query as a non-recombinant sequence belonging to the most frequent strain (B) its sensitivity and precision would be 41. 9% (this is the proportion of sites from strain B in the queries).
Thus, jpHMM's site-wise accuracy measures are only partially better than those of random classifiers. On the other hand the site-wise precision of SHERPAS, especially in the full version, is only marginally a ected (cf. Table  4 ). The site-wise sensitivity is lowered, which is due to the fact that error-rich regions are often unassigned. SHERPAS is also more accurate than jpHMM in terms of mosaic measures (cf. the frequencies of matches and mismatches   5. Accuracies observed on the dataset of simulated Nanopore HIV reads. For jpHMM only the results of launching it with its default options for HIV are reported, as the use of the -Q blat (fast) option resulted in the program failing to execute. All abbreviations are as in Table  2. Finally, once again SHERPAS-full is consistently more accurate than SHERPAS-reduced.
SHERPAS can be used for any dataset of viral sequences for which a reference alignment of su cient quality and size can be obtained. In fact SHERPAS, like SCUEAL, could also be used to detect recombinant bacterial sequences (Kosakovsky  Pond et al. , 2009) , although we have not experimented with such data. By default, SHERPAS uses the full pkDB, with ◊ F = 100 and window size 300. If the user chooses to run SHERPAS-reduced, the default parameters are ◊ R = 0. 99 and again window size 300. These default settings were chosen while trying to achieve a good balance among all accuracy measures, and assuming a volume of data that is not prohibitively large. However, users should be aware that the choice of settings will depend on the nature of the data and the goal of the analysis. For example, SHERPAS may be used as a first screen to detect potential recombinants in a large set of sequences.
To provide further insight into the ability of detecting evidence of recombination, we re-analyzed the queries in the HIV-pol dataset, which include a substantial number of sequences that are not interstrain recombinants. We then considered SHERPAS as a binary classifier for inter-strain recombination, classifying a sequence as a "positive" if at least one breakpoint is detected, and a "negative" otherwise. This allows us to observe how changing settings in SHERPAS (full vs. reduced database, threshold and window size) a ects its ability of recovering true positives (known as the recall of the classifier), and how much specificity has to be traded to improve this ability. The results of this experiment, shown in Fig. 2 , confirm that SHERPAS can indeed achieve high recall with the use of low thresholds (not the default one), and small windows, although users must be aware that this entails a loss of specificity.
Scaling-up Another important factor influencing how SHERPAS should be run is the amount of data to analyse. The query datasets that we used here were of relatively manageable sizes, to facilitate comparisons with slower software. Should the data to analyse be substantially more abundant (e. g. millions of reads), running SHERPAS in reduced mode may become more appealing, or even necessary in some cases. This is especially true if the reference tree is large, as in this case the speedup for SHERPAS-reduced is more pronounced (see  Sec. 4. 1). Moreover, for some datasets or applications, the accuracy of SHERPAS-reduced may be comparable to that of SHERPAS-full (see, e. g. , Sec. 4. 3 and Fig. 2 ). Because the running times of SHERPAS scale linearly with the amount of data to analyse, running SHERPAS on few millions queries is feasible in a matter of days (using SHERPAS-full) or in a matter of hours (using SHERPAS-reduced; see Table  1 ).
To the best of our knowledge, none of the recombination detection tools currently available are scalable to datasets of that size. Although our experiments focused on comparing SHERPAS to jpHMM -for the reasons detailed in Sec. 3. 1-previous comparisons of running times between jpHMM and phylogeny-based tools for recombination detection (namely REGA and SCUEAL) showed that jpHMM was at least as fast as those tools (see  Pineda-Peña et al. (2013) , Table  4 ), when run with the fast -Q blat option, meaning that the running time advantage of SHERPAS likely extends to the other available tools.
Accuracy Consistent with previous literature  (Schultz et al. , 2006; Kosakovsky Pond et al. , 2009; Schultz et al. , 2009) , we evaluated the predictive accuracy of SHERPAS using large datasets of semiartificial recombinant sequences, which combine fragments of real HIV and HBV sequences (the pre-queries) via artificially-introduced breakpoints. In the absence of large datasets of real sequences for which the true recombinant structure is known with certainty, this is a good way to evaluate a new method for recombination detection. (Note that we ensure that none of the pre-queries belongs to the reference alignment for the tested methods. ) Using sequences that are fully simulated using a fixed evolutionary model  (Kosakovsky Pond et al. , 2009)  is also a viable option, but the choice of the simulation parameters can have an important impact on the results, and the advantage over using semi-artificial sequences is unclear.
The experiments were designed to assess accuracy loss in SHERPAS compared to jpHMM  (Schultz et al. , 2006 (Schultz et al. , , 2009)). The choice of jpHMM is motivated in Sec. 3. 1. The other goal of our experiments was to explore the influence of the parameters of SHERPAS, including the use of full/reduced pkDB. Using two possible values for all parameters allows us to gauge their impact. The two window sizes (300 and 500 bp) were chosen on the basis of common practices of sliding window approaches (e. g. the REGA tool employs a window of 400 bp). Possibly the most interesting result here is the inferior performance of jpHMM compared to SHERPAS on the simulated Nanopore reads dataset. We suspect that the reason for this is that profile HMMs may be strongly a ected by large indels (which are common in these reads) and by errors that do not correspond well to their emission probabilities (which were estimated on error-free datasets by  Schultz et al.
Conclusion SHERPAS achieves a reasonable accuracy compared to state-of-theart inter-strain recombination detection tools for viruses, but is orders of magnitude more e cient. This advantage derives from the fact that SHERPAS does not need to align the query sequences, and from the relative simplicity of its classification algorithm. To the best of our knowledge, it is the first software that can estimate the recombinant structure of thousands of long sequences (up to whole genomes) within minutes or even seconds. It also appears to be relatively robust to high error rates typical of long read sequencing technologies. SHERPAS paves the way to systematic screening of recombinants in large datasets of long reads or assembled genome sequences. For completeness, here we describe in detail what phylo-k-mers are, and the main ideas behind their construction.
The choice of the software to use as well as the nucleotide substitution model and its parameters ◊, can be controlled with suitable options of the phylo-k-mer construction software (currently a module within RAPPAS). The result of this step is a large collection of tables containing all the marginal posterior probabilities of A Õ u,i that are relevant for the calculations that follow. See Figure  2  for an example of such a table, and its use.
5 by default). The phylo-k-mer database construction step relies on algorithms that generate all such phylok-mers, and stores them together with a list of (1) all the branches in E(T 0 ) for which Â P[w|x] is larger than the threshold, and (  2 ) their respective probability scores Â P[w|x] (see next section for more details). A number of alternative algorithms for this step have been the subject of recent research which will be described in an upcoming publication (N. Romashchenko et al. , pers. comm. ).
The full pkDB The full phylo-k-mer database (pkDB) is a data structure containing all phylo-k-mers and their associated information. It is a look-up table B that, given a phylo-k-mer w, returns the list of pairs B(w) = Ë (x 1 , s 1 ), (x 2 , s 2 ),. , (x |B(w)| , s |B(w)| ) È where the x i are branches of the reference tree (hence the use of the letter B) and s i = log Â P[w|x i ]. Unless otherwise stated, all logarithms are base 10. The pkDB also contains all the relevant information regarding the parameters that were used to generate the phylo-k-mers, most notably the threshold on their scores. In the following we call Thr B the threshold for log-probability scores, meaning that all s i in a B(w) list are such that s i > Thr B. The value of the threshold is taken into account in the calculations performed by SHERPAS: anytime a branch x does not appear in B(w), SHERPAS assumes log Â P[w|x] = Thr B.
The algorithm at a glance Below, we provide a detailed description of the algorithm underlying SHERPAS, complementing the high-level description in the main text. In order to understand the pseudocode that follows, here we introduce the main important ideas, namely the mathematical scores that SHERPAS computes. At any point during the execution of SHERPAS, only a subset of the branches of the reference tree are "active". These are the branches that are associated by the pkDB to at least one k-mer in the current window. We will denote this subset of branches by L, for consistency with the algorithm used by RAPPAS for placement  [11]. As the sliding window moves along, SHERPAS updates L, and for every branch x currently in L, it also keeps a score S[x] equal to the sum of all the log-probabilities associated to x by the k-mers in the current window.
Converting scores into likelihoods To compare the scores of di erent branches, these scores must first be converted into numbers that can be interpreted as likelihoods or probabilities. Recall the definition of S[x]: S[x] = |W | ÿ i=1 log Â P[w i |x] Because Â P[w i |x] is defined as the product of the probabilities of the nucleotides in w i at a given placement, the log-probability of each nucleotide in the sum above is usually counted k times (the only exception being the nucleotides near the ends of the window). Thus it makes sense to: (i) divide S[x] by k, and then (ii) convert the resulting log-probability into a probability. This leads to the following definition: ¸x = b S[x]/k , ( 5 ) where b is the base of the logarithm (10 in current databases). We interpret ¸x as an approximation of the likelihood of x being the phylogenetic origin of the sequence in W.
Second, sometimes entire strains may be composed of recombinant sequences. This occurs when a recombination event was ancestral to a common ancestor of all the sequences in the strain. This is believed to be the case for circulating recombinant forms (CRFs), which are recombinant forms that have been observed in several unrelated individuals. A naïve approach to include sequences of a particular CRF in a reference alignment is to define a strain coinciding with that CRF. This is what we have done in the two HIV datasets for CRF01_AE, which is believed to be a recombinant between subtype A and a now extinct (or unsampled) subtype E. Setting a strain to a CRF allows users to recognize novel recombinants between that CRF and other strains. However, the naïve approach above comes at a risk: the true evolutionary history of the reference sequences involves at least one reticulation ancestral to the CRF, meaning that the reference tree -whatever it is-will definitely not be correct in the part ancestral to that CRF.