dataesr/scientific-paragraphs-categorization huggingface dataset:
Although accurate, these extremely short corrected reads could thus not yield a proper assembly on the A. baylyi dataset, and could not be assembled at all on the S. cerevisiae. Apart from LoRMA, the lowest error rate was reached by Canu on the A. baylyi dataset, with 5. 4%, and by MECAT on the S. cerevisiae dataset, with 19. 9%. In terms of assembly, and apart from LoRMA, all tools performed well on the A. baylyi dataset, except for Canu, which only reached 93. 6% of genome coverage. However, on the S. cerevisiae dataset, all tools displayed unsatisfying results, reaching greater number of contigs and smaller genome coverage than most hybrid correction tools. More precisely, the highest genome coverage was reached by CONSENT , with only 36. 9%. In terms of runtime, all hybrid correction tools but LoRDEC and Jabba performed slower than self-correction methods.
It is however worth noting that Jabba was the fastest tool, requiring at most 7 minutes to perform correction on the S. cerevisiae dataset. The largest runtime was reached by NaS, requiring more than 94 hours on the A. baylyi dataset, and more than 16 days on the S. cerevisiae, leading to the experiment being stopped before it finished. In comparison, the slowest self-correction tool only displayed a runtime of 1. 5 hour on the S. cerevisiae dataset. Conversely, in terms of memory consumption, self-correction methods were more demanding than most hybrid correction methods. This can especially be seen with LoRMA requiring almost 32 GB on both datasets. Furthermore, on the S. cerevisiae dataset, only CoLoRMap and Hercules displayed a significantly higher memory consumption than other self-correction tools, LoRMA excluded.
These results thus suggest that, even with sequencing depth as high as 100x, self-correction methods cannot perform well on extremely noisy datasets, reaching error rates of 30% and more, which was common in first long-read sequencing experiments. Although more time consuming, hybrid correction methods should thus be preferred on such data, since they do manage to properly perform correction, efficiently lower the error rates of the reads, and even lead to satisfying assembly results. Corrected long reads were obtained from the Genoscope website. 3 Canu stopped and reported an error during the correction of the S. cerevisiae dataset. 4 Daccord could not perform correction on the S. cerevisiae dataset, since the alignment step with DALIGNER required more than 128 GB of memory. 5 LoRMA corrected reads could not be assembled for the S.
High error rate, low coverage Here, we present results on the C. elegans real dataset of Table  3. For this dataset, when possible, we evaluated the split corrected long reads. When split reads were not available, we evaluated the trimmed corrected long reads. Finally, when both split and trimmed reads were not available, we evaluated the complete corrected long reads. These results are summarized in Table  5  and commented below. Moreover, for these experiments, we excluded the following tools from the comparison, for the ensuing reasons: • HALC, because it stopped and reported an error during correction; • LSC, because of its large runtimes on previous experiments; • Nanocorr, because of its large runtimes on previous experiments; • NaS, because of its large runtimes on previous experiments; • Daccord , because it could not perform correction, since the alignment step with DALIGNER required more than 128 GB of memory; • FLAS , because it stopped and reported an error during correction; • MECAT , because it stopped and reported an error during correction.
On this dataset, conclusions that were drawn in Section 3. 3 are further confirmed. Indeed, since the error rate remained high, but the sequencing depth only reached 20x, self-correction methods performed worse than previously. In particular, both FLAS and MECAT could not process this dataset, and reported an error during correction. Moreover, CONSENT , the best-performing self-correction method on this dataset, only managed to reduce the error rate to 8. 3%. These results are in accordance with the fact self-correction tools only rely on long reads, and thus necessitate larger sequencing depths to perform efficient correction. On the other hand, hybrid correction tools, for which the sequencing depth of the long reads has no impact on correction, still performed well. However, since the studied genome was more complex than those of Section 3.
However, all assemblies were of lower quality than the ones presented in Section 3. 3, which can be explained by the lower sequencing depth of the long reads. In terms of runtime, hybrid correction tools were still much slower than self-correction tools. Indeed, Canu, which was the slowest self-correction tool on this dataset, only required 14 hours to run, while hybrid correction tools such as CoLoRMap, ECTools, and HG-CoLoR required more than 80 hours. However, in the same fashion as in Section 3. 3, hybrid correction tools LoRDEC and Jabba were the fastest, both only requiring around 1 hour to run. In terms of memory consumption, all hybrid and self-correction tools were comparable. However, the least demanding tools were the hybrid correctors ECTools and LoRDEC, which required, respectively, only 5 GB and 2 GB, while Canu, the least demanding self-correction tool, required 9 GB.
On the other hand, most demanding tools were CoLoRMap, Hercules and LoRMA, which all required around 32 GB. These results suggest that, when the error rate of the reads is high and the sequencing depth is low, self-correction methods cannot perform well. These observations are in accordance with those of Section 3. 3, indicating that self-correction tools cannot perform well on high error-rates datasets, even with high coverage depth. Logically, lowering the coverage of the reads, thus only worsens the results provided by self-correction tools. As a result, although they are more time consuming, hybrid correction methods should be preferred on such data, since they do manage to properly perform correction, and since the sequencing depth of the long reads has no impact on their performances.
Medium error rate, low coverage Here, we present results on the A. baylyi 20x, E. coli 20x, S. cerevisiae 20x, C. elegans 20x and E. coli real datasets of Table  3. For these datasets, when possible, we evaluated the split corrected long reads. When split reads were not available, we evaluated the trimmed corrected long reads. Finally, when both split and trimmed reads were not available, we evaluated the complete corrected long reads. These results are summarized in Table  6  for simuated data, and in Table  7  for real data. Results on simulated data are commented in Section 3. 5. 1, and results on real data are commented in Section 3. 5. 2.
Simulated data On these datasets, most hybrid correction tools managed to reduce the error rates to around 1% or less, on the A. baylyi, E. coli and S. cerevisiae datasets. In the same fashion as in Section 3. 3, only LSC and Hercules displayed poor performances on these three datasets, displaying error rates up to 6. 1% for the former, and up to 10. 9% for the latter, on the S. cerevisiae dataset. Moreover, it is worth noting that NaS, which performed very well during the experiments of Section 3. 3, did not manage to reduce the error rates of the reads below 1% on any of these datasets, and even reached error rates up to 2. 7% on the S. cerevisiae datasets. Another interesting point is that all hybrid methods displayed performance drops on the C. elegans dataset, even though it displayed the same error rate as all the other datasets.
On this dataset, only HG-CoLoR, Jabba and Proovread managed to reduce the error rate below 1%. As explained in Section 3. 4 this can be explained by the fact that the genome of C. elegans is more complex and contains more repetitions than the genomes of the other species studied here. As for self-correction, although the error rates of the original datasets have been divided by 1. 5-2. 4 compared to the experiments of Section 3. 3 and Section 3. 4, most methods still displayed poor performances. Indeed, of all the self-correction methods, only Daccord and MECAT managed to reduce the error rates of the reads below 1%. However, due to its high memory requirements, Daccord could not scale to the C. elegans dataset. Moreover, compared to hybrid correction methods, MECAT reported a significantly smaller number of bases, and thus corrected less reads.
LoRMA performed well in terms of reduction of the error rates as well, but as previously mentioned, aggressively split the reads, as underlined by the trimmed / split reads and mean missing size metrics. Moreover, on the C. elegans dataset, most self-correction methods did not display the same performance drop as hybrid correction methods, apart from FLAS and LoRMA. This underlines the fact that self-correction seems to be more suited for more complex, repeat-rich, genomes. Moreover, it is also interesting to note that metrics reported by ELECTOR provide additional insight as to how the different tools behave. For instance, as previously mentioned, they show that LoRMA tends to split a vast majority of the reads, and that large proportions of the original reads are missing from these split corrected reads.
In terms of runtime, all hybrid correction tools but LoRDEC and Jabba once again performed slower than self-correction methods. On all these datasets, the fastest tool was MECAT , requiring as little as 22 seconds to perform on the A. baylyi dataset, and at most 18 min to perform on the C. elegans dataset. The slowest correction tool was once again NaS, requiring up to 9 days to perform on the S. cerevisiae dataset. Comparatively, the slowest self-correction tool, Canu, only required 4 h 37 min to correct the larger C. elegans dataset. In terms of memory consumption, self-correction tools were once again generally more demanding than hybrid correction methods, CoLoRMap and Hercules excluded, on the A. baylyi, E. coli and S. cerevisiae datasets. However, on the C. elegans dataset, a greater number of hybrid correction tools displayed higher memory requirements than self-correction tools.
CONSENT , the most memory-consuming self-correction tool after LoRMA, required 14. 5 GB, and was indeed comparable to Jabba, whereas other hybrid tools such as Proovread and HG-CoLoR displayed a memory peak of 20 GB. These results suggest that, even when error rates reach less than 20%, self-correction methods cannot perform well on low coverage datasets. In fact, only MECAT and Daccord could provide satisfying correction on these datasets, but the latter was limited by its high-memory requirements, and could not scale to the larger C. elegans dataset, limiting its application. Hybrid correction thus remains more suited for such data, although it can be slower and more memory consuming. However, it is also worth noting that these hybrid correction methods tend to see their performances drop as the complexity and the number of repeats of the studied organisms grow.
This is not the case for self-correction methods, and a tool such as MECAT could thus be preferred when studying high-complexity organisms. However, it outputs less corrected reads than most hybrid tools, and might thus not be suited for all types of downstream analyses, including assembly. 3. 1 Nanocorr was not launched on the C. elegans dataset due to its large runtimes. 2 NaS was not launched on the C. elegans dataset due to its large runtimes. 3 Daccord could not perform correction on the C. elegans dataset, since the alignment step with DALIGNER required more than 128 GB of memory.
Real data As on the previous datasets of Section 3. 5. 1, almost all hybrid correction tools managed to reduce the error rate below 1% on this dataset. However, once again, LSC and Hercules displayed poor performances, reaching error rates of 8. 8% for the former, and up to 14. 2% for the latter. On this dataset however, NaS once again performed well. This can be explained by the fact that it was designed for and tested on ONT data, and is thus probably biased towards better ONT data correction. In terms of assembly, all tools but LSC, Jabba and ECTools, the latter not being able to yield any assembly, resulted in a single-contig assembly, covering a large proportion of the reference genome, the smallest being of 97. 78% for LoRDEC. Same conclusion as Section 3. 5. 1 also apply to self-correction tools on this dataset.
On this dataset, the fastest tool was Jabba, requiring 2 min, followed by MECAT , which required 4 min. This underlines the fact that unlike on PacBio data, MECAT tends to be slightly slower when correcting ONT reads. Conversely, the slowest correction tool was NaS, requiring 3 days to perform correction, while Canu, the slowest self-correction tool only required 35 min. In terms of memory consumption, both hybrid and self-correction tools were comparable, although hybrid tools such as CoLoRMap, and Proovread, and self-correction tools such as Daccord and LoRMA displayed higher memory requirements, the latter once again being the most memory-consuming tool, requiring a whole 32 GB. These results further confirm conclusions drawn in Section 3. 5. 1, and once again suggest that self-correction methods cannot perform well on low coverage datasets, when error rates reach around 20%.
Moreover, on this real ONT dataset, MECAT did not perform as good as on the simulated PacBio datasets of Section 3. 5. 1, both in terms of reduction of the error rate and in terms of quality of the assembly. These results thus reinforce the previous conclusions, confirming MECAT is not best suited for downstream analyses such as assembly, but also showing MECAT seems to be less efficient when dealing with ONT data. As a result, hybrid correction once again proves to be more efficient for handling such data, although it can be slower and more memory consuming. This corresponds to the E. coli real dataset of Table  3. 1 ECTools corrected reads could not be assembled. 2  LoRMA corrected reads could not be assembled.
Medium error rate, medium coverage Here, we present results on the A. baylyi 60x (medium), E. coli 60x (medium), S. cerevisiae 60x (medium) and C. elegans 60x (medium) datasets of Table  3. For these datasets, when possible, we evaluated the split corrected long reads. When split reads were not available, we evaluated the trimmed corrected long reads. Finally, when both split and trimmed reads were not available, we evaluated the complete corrected long reads. These results are summarized in Table  8  and commented below. Moreover, for these experiments, we excluded the following tools from the comparison, for the ensuing reasons: • ECTools, because of its unsatisfying results on previous experiments; • Hercules, because of its unsatisfying results and large runtimes on previous experiments; • LSC, because of its unsatisfying results on previous experiments; • Nanocorr, because of its large runtimes on previous experiments; • NaS, because of its large runtimes on previous experiments; On these datasets, as in Section 3.
5. 1, all assessed hybrid correction tools managed to reduce the error rates to around 1% or less, on the A. baylyi, E. coli and S. cerevisiae datasets, and hybrid methods that were ran of the C. elegans dataset displayed performance drops, especially FMLRC, which reached an error rate of 3. 42%. However, for self-correction, unlike in Section 3. 5. 1, most tools did manage to reduce the error rates to around 1% on all four datasets. Actually, only CONSENT reached an error rate of more than 1% on all datasets, and only CONSENT and FLAS reached an error rate of more than 2% on the C. elegans dataset. Once again, Daccord performed best on the first three datasets, but could not be run on C. elegans due to its memory requirements, and LoRMA reported a large proportion of split reads. Despite this fact, most self-correction methods nonetheless reported less corrected bases than hybrid methods, and LoRMA and MECAT especially performed poorly regarding this metric.
Moreover, it is worth noting that, compared to the results of Section 3. 5. 1, for which the error rates were the same, self-correction methods performed better on these datasets, for which coverage was higher. This underlines the fact that, with sufficient coverage, self-correction methods are well suited for the processing of moderately high error rates datasets, especially given the fact that, apart from FLAS and LoRMA, they do not suffer from performance drops when applied to complex and repeat-rich genomes. In terms of runtime, even though the slowest methods have been excluded, all hybrid correction tools except Jabba once again performed slower than self-correction methods. On all datasets, the fastest tool was once again MECAT , requiring 2 min to perform on the A. baylyi dataset, and at most 1 h 37 min to perform on the C.
elegans dataset. Conversely, the slowest tool was Proovread, requiring almost 18 hours to perform on the S. cerevisiae dataset. In comparison, LoRMA, which was the slowest self-correction tool on this dataset, only required a little less than 2 hours to perform correction. In terms of memory consumption, self-correction tools were once again more demanding than most hybrid correction tools on all datasets, CoLoRMap and Proovread excluded. However, it is worth noting that hybrid correction tool Jabba displayed an important peak in memory consumption on the C. elegans dataset, requiring more than 13 GB, while it only required around 1 GB on the three other datasets. On this dataset, Jabba was thus comparable to or more demanding than most selfcorrection tools. These results suggest that, when error rates reach 20% or less, self-correction methods can perform well as long as the coverage is sequencing depth is sufficiently high, and that a sequencing depth of around 60x is sufficient.
In terms of quality of the results, self-correction methods are comparable to hybrid correction methods on such datasets. Moreover, these results also show that self-correction methods usually perform faster than hybrid correction methods, although they require larger amounts of memory. These observations are particularly interesting, given the additional fact that, unlike hybrid correction, self-correction methods do not see their performances drop as the complexity of the studied organisms grows. As a result, although hybrid correction remains efficient, self-correction is also well suited for such data, and can even be preferred when studying complex organisms, or when fast processing is needed. This corresponds to the A. baylyi 60x (medium), E. coli 60x (medium), S. cerevisiae 60x (medium) and C.
elegans 60x (medium) datasets of Table  3. 1 CoLoRMap was not launched on the C. elegans dataset due to its large runtimes. 2 HG-CoLoR was not launched on the C. elegans dataset due to its large runtimes. 3 Proovread was not launched on the C. elegans dataset due to its large runtimes. 4 Daccord could not perform correction on the C. elegans dataset, since the alignment step with DALIGNER required more than 128 GB of memory.
Low error rate, low coverage Here, we present results on the E. coli 30x, S. cerevisiae 30x and C. elegans 30x datasets of Table  3. For these datasets, when possible, we evaluated the trimmed corrected long reads. When trimmed reads were not available, we evaluated the complete corrected long reads. These results are summarized in Table  9  and commented below. Moreover, for these experiments, we excluded the following tools from the comparison, for the ensuing reasons: • ECTools, because of its unsatisfying results on previous experiments; • Hercules, because of its unsatisfying results and large runtimes on previous experiments; • LSC, because of its unsatisfying results on previous experiments; • Nanocorr, because of its large runtimes on previous experiments; • NaS, because of its large runtimes on previous experiments; On these datasets, both hybrid and self-correction methods, except LoRMA, did manage to reduce the error rate to around 1% or less, on all datasets.
More precisely, only the hybrid methods FMLRC, HG-CoLoR and LoRDEC reached an error rate of more than 1% on the C. elegans dataset. In comparison, on this dataset, all selfcorrection tools but LoRMA managed to reduce the error rate below 1%. This observation is in accordance with that previously made in Section 3. 5. 1 and Section 3. 6, indicating that self-correction methods usually perform better on complex, repeat-rich organisms, compared to hybrid correction methods, which usually suffer from performance drops. Moreover, it is interesting to note that, on these datasets, self-correction tools such as FLAS , CONSENT and Daccord reported as many corrected bases as hybrid methods, although LoRMA and MECAT still performed poorly regarding this metric. This is especially interesting, since it shows selfcorrection methods can not only provide high-quality correction on such datasets, but also manage to correct a large number of reads.
Overall, hybrid correction and self-correction methods were thus highly comparable on these three datasets, in terms of quality of the results. This underlines the fact that, even with low coverage, self-correction methods do manage to perform well on datasets for which the error rate reaches around 12%. In terms of runtime, all hybrid correction tools except Jabba once again performed slower than self-correction methods. More precisely, Jabba was even the fastest tool on all these datasets, requiring at most 43 min to perform on the C. elegans dataset. MECAT , the fastest self-correction tool nonetheless also performed correction fast, since it only required 48 min on that same dataset. Jabba excluded, however, self-correction methods performed faster than most hybrid correction methods. This can especially be seen on the C.
elegans dataset, where CONSENT and Canu, the two slowest self-correction tools, both required a little more than 9 hours to perform, while hybrid correction tools such as CoLoRMap and HG-CoLoR required respectively 4. 5 and 6 days to perform. In terms memory consumption, self-correction tools were more demanding than most hybrid correction tools on all datasets. However, hybrid correction tools such as CoLoRMap, HG-CoLoR and Proovread required more memory than most self-correction tools, LoRMA excluded, on the C. elegans dataset. These results thus suggest than, when error rates reach around 12% or less, self-correction methods can be satisfyingly applied, even when the sequencing depth only reaches 30x. Indeed, both self-correction and hybrid correction methods were comparable in terms of quality of the results, on all of these datasets.
Moreover, these results also show that self-correction methods usually perform faster than hybrid correction methods, and that their memory consumption can also be lower on large datasets. These observations are especially interesting, since they suggest that, given the error rate of the reads is low enough, self-correction methods can provide a high-quality correction, even on low coverage datasets, and can thus allow to avoid the cost of having to sequence both long and short reads to perform efficient analysis. As a result, this suggests that self-correction is best suited, and should thus be preferred when analyzing such low error rates and low coverage datasets.