dataesr/scientific-paragraphs-categorization huggingface dataset:
In contrast, HmmCleaner was more specific for eukaryotic than prokaryotic sequencs (~94. 5% versus ~87%), our eukaryotic genes being less divergent. The same was true for PREQUAL, but to a smaller extent (~94. 5% versus ~92. 8%). In summary, sensitivity and specificity of both segment-filtering methods were relatively unaffected by error type and data type. These results were especially welcome for HmmCleaner, which had been trained on prokaryotic frameshifts alone. Segment-filtering methods were developed based on the hypothesis that block-filtering methods are not adapted to detect primary sequence errors. To formally test this assumption, we confronted PREQUAL and HmmCleaner to the block-filtering software BMGE and to OD-seq  [24]  and GUIDANCE2  [12] , two methods designed to filter outlier sequences from MSAs.
Sources of HmmCleaner false positives As shown in Fig. 3 , specificity of HmmCleaner is lower than its sensitivity and is also more variable across MSAs. When genuinely homologous segments are highly divergent, i. e. , display a weak similarity to other sequences of the MSA, false positives are unavoidable. Accordingly, specificity was higher in UARs (Fig. 2 ) than in AARs (Additional file 1 Figure  S1 ). A refined analysis shows that HmmCleaner specificity decreases with the gap frequency (Additional file 1 Figure  S7A ), the evolutionary rate (Additional file 1 Figure  S7B ) and the fraction of AARs (Additional file 1 Figure  S7C ). This confirms that its low specificity is due to evolutionary divergence. Such a negative correlation between sequence divergence and HmmCleaner specificity can be due to: (i) the presence of overlooked primary sequence errors in our datasets, (ii) the presence of alignment errors that would result in detection errors, (iii) the detection of segments corresponding to insertion events, or (iv) the detection of homologous but divergent segments that look like primary sequence errors (see above).
Detection of positive selection in the presence of primary sequence errors Having addressed the efficiency of HmmCleaner at dealing with simulated primary sequence errors, we can now assess its usefulness on empirical evolutionary inferences, such as detection of positive selection, topological accuracy and inference of branch lengths. The presence of a primary sequence error in a MSA (i. e. , a highly divergent segment) is expected to severely increase the number of non-synonymous substitutions, thereby creating a strong signal for positive selection in the branch leading to the corresponding sequence. To test this idea, we simulated an out-of-frame segment of length 10 to 50 aa into 116 MSAs from the OrthoMAM database, before realigning the MSA (MAMMALIA dataset, see Materials and Methods).
Over 10 replicates of the simulation, branch-specific positive selection detected by the standard likelihood ratio test method of Nielsen and Yang  [26, 27]  increased from 8. 28 to 95. 69% (Table  3 ). The few cases with non-significant likelihood ratio test results corresponded to shorter erroneous segments (~20 versus ~30 aa), which could have been introduced into divergent regions. As expected, a primary sequence error generates a strong erroneous signal of positive selection. The use of HmmCleaner on the 116 raw MSAs decreased branch-specific positive selection from 8. 28 to 3. 88% (Table  3 ). A detailed manual analysis of the MSAs in which the signal for positive selection had disappeared generally found the presence of structural annotation errors that were correctly detected and removed by HmmCleaner.
The remaining 3. 88% of significant likelihood ratio test results may correspond to real positive selection or to structural annotation errors not detected by HmmCleaner. Importantly, the use of HmmCleaner on the MSAs in which we had introduced primary sequence errors drastically reduced the detection of positive selection (7. 76%). This value was slightly higher than the control (3. 88%) and likely due to incomplete removal of the errors by HmmCleaner, in agreement with the sensitivity estimated above (Figs. 3 4 ). PREQUAL performed similarly to HmmCleaner but was less efficient at reducing the detection of positive selection after error insertion (13. 62%), in agreement with its Fig. 5  Mean pairwise identity of the low similarity segments (LSSs) detected on 400 raw MSAs (no simulation) by HmmCleaner used with the default scoring matrix lower sensitivity.
Relative effect of primary sequence and alignment errors on phylogenetic inference To evaluate the relative importance of primary sequence errors and alignment errors on real data, we compared the effect of HmmCleaner, PREQUAL and two block-filtering software (BMGE and TrimAl) on the accuracy of phylogenetic inference. We also examined filtering methods that reduce the stochastic (sampling) error (removal of partial sequences and selection of the longest genes), as this type of error might be critical for single-gene inferences. Two aspects of phylogenetic inference were considered: tree topology and branch lengths. Two datasets of orthologous genes for which the correct species phylogeny is reasonably well established were used: (1) 14,261 genes from mammals, obtained exclusively from genomic data  [29] , named MAMMALIA, (2) 4593 genes from vertebrates, obtained mainly from transcriptomic data  [30] , named VERTEB RATA.
Our expectation was that primary sequence errors would be more frequent in MAMMALIA than in VER TEBRATA due to incorrect structural annotations of genomic data  [31]. Phylogenetic accuracy as measured by the frequency of correctly recovered clades was computed for various conditions (Table  4 ). Major improvements were observed in three cases: (i) inferences based on nt MSAs over those based on aa MSAs with MAMMALIA (63. 19% versus 45. 29%), (ii) removal of partial sequences (< 100 aa) in VERTEBRATA (68. 70% versus 65. 64%) and (iii) use of the longest genes (+ 6. 58%, + 8. 30% and + 3. 21% for nt and aa MAMMALIA and aa VERTEBRATA, respectively). All these cases actually correspond to a reduction of the stochastic error, either due to (i) the increased amount of information present in nt, (ii) the removal of sequences without enough signal to be accurately positioned, or (iii) the larger number of positions, respectively.
22% versus 5. 61%). Similarly, the better specificity of PREQUAL (Table  2  and Additional file 1 Table  S1 ) could explain its slightly higher accuracy (65. 73%) and the reduced amount of data removal (3. 06%). For the VER TEBRATA dataset, which likely contains few primary sequence errors, the removal of data is slightly deleterious (except for PREQUAL, with an improvement of 0. 09%), illustrating how precise data filtering should be. In contrast, the MAMMALIA dataset demonstrated the positive effect of using HmmCleaner on genomic-based datasets, which are more likely to contain annotation errors: accuracy improved from 63. 19% to 66. 30% for nt MSAs. BMGE and TrimAl also increased accuracy, but less than HmmCleaner (63. 59% and 63. 77%, respectively), while the random removal of characters expectedly decreased accuracy (62.
Accordingly, for VERTEBRATA, the combination of the two types of filters show the lowest accuracy among all our analyses. In contrast, for MAMMALIA, the accuracy increased when both filters were applied versus when a single one was used: from ~64. 7% to ~66. 5% (nt) and ~45. 5% to ~46. 5% (aa). The comparison of segment+block-filtering with segment-filtering was more ambiguous: the accuracy increased for nt MSAs (from 66. 29% for HmmCleaner to 66. 56% for HmmCleaner+BMGE, the best accuracy observed for MAMMALIA) but decreased for aa MSAs (from 46. 98% for HmmCleaner to 46. 52% for HmmCleaner+BMGE). These contrasted results illustrate the difficulty of data filtering, data loss increasing stochastic error while decreasing reconstruction errors. In conclusion of this section, HmmCleaner is more efficient than BMGE and TrimAl, and to a lesser extent than PREQUAL, at improving topological accuracy for genomebased MSAs, whereas filtering methods slightly decrease accuracy for transcriptome-based MSAs.
As the genes under study were orthologous, the correlation coefficients were expected to be high  [32] , fluctuating only because of stochastic sampling noise and heterotachy  [34]. For both MAMMALIA and VERTEBRATA aa datasets (Fig. 6 ), the average correlation coefficients were 0. 662 and 0. 710, respectively, but 0. 778 for MAMMALIA nt MSAs. Again, stochastic error due to the loss of information generated by translation appears as a key factor. In contrast to topological accuracy, the improvement in correlation provided by HmmCleaner was similar for the three datasets (aa VERTEBRATA: 0. 066, aa MAMMALIA: 0. 089, nt MAMMALIA: 0. 079). Interestingly, for MAMMALIA, the average correlation coefficient for cleaned aa MSAs was similar to the one of raw nt MSAs (0. 749 and 0. 778). PREQUAL performed similarly to HmmCleaner, but was slightly less efficient, even for VERTEBRATA (Fig.
6 ), likely because of its lowest sensitivity. In sharp contrast, block-filtering methods (BMGE and TrimAl) had virtually no impact, even when applied after HmmCleaner. Segment-filtering methods seem thus to be more efficient than block-filtering methods to remove primary sequence errors affecting branch-length estimates. This is in agreement with our hypothesis that primary sequence errors, which are the target of HmmCleaner and PREQUAL, are more detrimental to evolutionary inferences than alignment errors, which are the target of BMGE and TrimAl. Finally, we examined the effect of filtering software on the branch lengths of the concatenated trees. All pairwise comparisons (e. g. , BMGE versus HmmCleaner) yielded correlation coefficients > 0. 98. Interestingly, for   S2  for mean values VERTEBRATA, a few outliers were identified, corresponding to shortened branch lengths in HmmCleanerbased phylogenies for species from which genomic data had been used (e.
Conclusion In this article, we presented a new version of HmmCleaner, a software package that automatically identifies and removes low similarity segments in MSAs with the purpose of limiting the negative effect of primary sequence errors on evolutionary inferences. The performance of our method was investigated through analyses of both simulated and empirical data. HmmCleaner shows an excellent sensitivity to primary sequence errors ≥12 aa in length in simulations. Its specificity to simulated errors is also high, with its false positives mostly corresponding to insertions or low similarity segments that would be difficult to handle in subsequent steps of analysis. We showed that segment-filtering software (HmmCleaner and PREQUAL) have more positive effects on evolutionary inferences (detection of branch-specific positive selection, topological accuracy and branch-length estimation) than the commonly used block-filtering software (BMGE and TrimAl).
This suggests that primary sequence errors are more detrimental to evolutionary analyses than alignment errors. Therefore, we argue that the efforts of the research community should address both alignment and primary sequence errors, in other words that more energy should be devoted on structural annotations. In this respect, HmmCleaner proved to be efficient at pointing them out, being slightly more sensitive than the recently developed PREQUAL  [20]. Given the pervasiveness of primary sequence errors, we recommend the use of segment-filtering methods in high-throughput analyses of eukaryotic genomic data. On the long run, it would be interesting to evaluate whether HmmCleaner (or other equivalent toolssuch as PREQUAL) could replace block-filtering software. For now, HmmCleaner targets low similarity segments that are by essence difficult to align and therefore may decrease the frequency of alignment errors, possibly to the extent of making them negligible.
Dataset creation To optimize the parameters and study the performance of HmmCleaner, we created four datasets by assembling MSAs of protein-coding genes sampled from four different prokaryotic lineages (Alphaproteobacteria, Cyanobacteria, Euryarchaeota, and Crenarchaeota). We chose prokaryotes to minimize the presence of annotation errors, as these lineages are mostly devoid of introns, simplifying the structural annotation of their genes. Yet, a few structural annotation errors will likely subsist, in particular due to sequencing errors, incorrect start codon predictions, programmed ribosomal frameshifts and programmed transcriptional realignments  [35]. For each lineage, we retrieved annotated RefSeq genomes from NCBI FTP (61 Alphaproteobacteria, 195 Cyanobacteria, 42 Crenarchaeota and 179 Euryarchaeota) and used the corresponding proteomes to define orthogroups with OrthoFinder (E-value = 10e-5; inflation parameter = 1.
5)  [36]. In order to maximize the proportion of true orthologous sequences, only orthogroups with at least 75% of our taxon sampling and < 10% of multiple copies were selected. To minimize the sequence length heterogeneity in the orthogroups, we studied the length distribution of the clusters and kept only those having a mean sequence length ≥ 100 aa and with < 5% outliers in the sequence length distribution. Outliers were defined as sequences having a length shorter than the mean length minus 1. 96 times the standard deviation. In addition, we removed these outlier sequences from the retained orthogroups. To assemble the four final datasets of 100 MSAs each, we selected at random 100 orthogroups for each of the four lineages, and aligned their sequences with MAFFT 7. 309  [21]  (L-INS-i algorithm, 5000 iterations).
Since our simulations introduce frameshifts in nt sequences (see below), we transferred the alignment gaps from protein sequences to the corresponding nt sequences. To study the impact of HmmCleaner on evolutionary inferences, we used two additional datasets assembled from animal sequences. The first dataset (MAMMALIA) corresponded to the 14,261 orthologous genes with ≥50% of 43 species present from OrthoMAM v9  [29]. As both nt and amino-acid alignments were available for download, we used both types of sequences. In contrast, only amino-acid sequences were available for the second dataset (VERTEBRATA). The latter corresponded to the 4593 orthologous genes from Irisarri et al. [30]. Because these authors had used filtering softwares during their dataset construction, we had to re-apply their last step (selection of a single sequence per organism and construction of chimeric sequences when necessary) using SCaFoS  [37]  on a pre-filtering version of the corresponding MSAs.
Parameter optimization To optimize the four parameters of the scoring matrix of HmmCleaner, we simulated frameshift errors on the two large datasets (Cyanobacteria and Euryarchaeota). For each nt MSA, 4 subsets of sequences of different sizes (5, 10, 25 and 50 sequences) were drawn 100 times at random. On each of these samples, 1 to 5 sequences were randomly affected by a primary sequence error of length 10 to 100 aa. HmmCleaner was then run on each resulting amino-acid MSA (complete under 2835 different combinations of its four parameters. There were 9, 7, 9 and 5 possible values, respectively, for c1 (-0. 05 to -0. 25 by step of 0. 025), c2 (-0. 02 to -0. 08 by step of 0. 01), c3 (0. 05 to 0. 25 by step of 0. 025) and c4 (0. 4 to 0. 6 by step of 0. 05). These ranges were defined based on preliminary simulations aimed at thoroughly exploring the zone of high specificity and high sensitivity.
99, Additional file 1 Figure  S2C-D ). Third, we considered the potential impact of the number of sequences in the MSAs (5, 10, 25 or 50). In this case, we expected a larger effect owing to the dependence of pHMM statistical power on the amount of observations available to build the models. Specificity showed a high correlation (> 0. 95) while comparing MSAs with 5 sequences to those with 50 sequences (Additional file 1 Figure  S2F ). In contrast, sensitivity was more affected, and the correlation coefficient dropped to 0. 78 (Additional file 1 Figure  S2E ). In agreement with our intuition, sensitivity was always better with 50 than with 5 sequences. However, the parameter quartets leading to high sensitivity for 5 sequences were the same to yield high sensitivity for 50 sequences, indicating that the number of species does not much impact the optimization of HmmCleaner parameters.
Characterization of HmmCleaner performance To characterize the impact of the length and number of frameshift errors on HmmCleaner sensitivity and specificity, we ran simulations at 4 predetermined error lengths (10, 33, 66 and 100 aa) and with 4 different numbers of sequences affected  (1, 5, 10 or 15). For each of these 16 combinations, 100 simulations were performed on each of the 100 MSAs of the 4 lineages using subsets of 25 randomly drawn sequences. HmmCleaner was run on these 160,000 MSAs per prokaryotic lineage (640,00 MSAs in total) using the default scoring matrix and the complete strategy. Ten additional simulations were carried out in the same conditions (16 combinations of error characteristics on subsets of 25 sequences) to compare HmmCleaner with the four scoring matrix and PREQUAL.
PREQUAL was run with default parameters and without the removal of repeated regions. For the high-resolution analysis of the impact of the error length on sensitivity, the same type of simulation was run with only one sequence affected by a primary sequence error of length ranging from 1 to 33 aa. Additional simulations were carried out to expand the observations obtained on frameshift errors to the other types of primary sequence errors that our simulator can generate. At the same time, we extended our dataset to MSAs of eukaryotic species by selecting 112 alignments from the MAMMALIA dataset (MSA with > 25 sequences out of the 116 genes used for positive selection, see below) and 170 alignments from VERTEBRATA for which we retrieved nucleotide sequences (MSAs with less than 3 missing species).
For each type of errors (frameshifts, scrambled segments and arbitrary insertions), subsets of 25 sequences were drawn out of the complete MSA and 1 to 5 sequences were affected by an error of length between 10 to 100 aa. This simulation was run 100 times per MSA for HmmCleaner (default), BMGE (loose) and PREQUAL, and 10 times for OD-SEQ (default parameters) and GUIDANCE2 (default threshold to determine outlier sequences). To study the characteristics of the low similarity segments detected by HmmCleaner, so as to characterize the sources of its false positives, it was run with the default scoring matrix and the complete strategy on the raw MSAs from the four prokaryotic lineages, as MSAs of eukaryotic lineages are more likely to contain real primary sequence errors (mainly incorrect structural annotation).
Effect of HmmCleaner on evolutionary inferences Analyses of positive selection were performed on a subset of the MAMMALIA dataset. To reduce structural annotation errors, we selected the 446 nt MSAs with branch length R 2 above 0. 95 (computed as in Simion et al. 2017  [32] , see below). To limit the computational burden and to introduce errors of a sizable length representing only a few percent of the sequences, we selected the 116 MSAs between the first quartile and the median on the MSA width distribution. For each MSA, we simulated one primary sequence error of random length (10 to 50 aa) at a randomly chosen position of a randomly chosen sequence 10 times and aligned with MAFFT. For each simulation, we tested for positive selection in the affected branch for 10 versions of the corresponding MSA: (1) the original MSA, (2) the original MSA cleaned by 5 filtering software configurations (HmmCleaner, PREQUAL, BMGE and TrimAl), (3) the erroneous MSA, and (4) the erroneous MSA cleaned by 5 filtering software configurations (HmmCleaner, PREQUAL, BMGE and TrimAl).
Detection of positive selection was performed using a likelihood ratio test between two models  [26, 27] : model A, in which ω estimation is free (i. e. , allowing positive selection), and model B, in which ω is fixed to 1 (i. e. , no selection). Likelihood values for both models were obtained using codeml (both models: runmode = 0, method = 0, clock = 0, model = 2, CodonFreq = 2, NSsites = 2, fix_kappa = 0, kappa = 2; model A: fix_omega = 0, omega = 0. 2; model B: fix_omega = 1, omega = 1). Positive selection was considered present when the likelihood ratio test between models A and B returned a value > 13. 82 (Chi-square critical value for alpha = 0. 001 and 2 degrees of freedom). To test the effect of filtering methods on the accuracy of single-gene phylogenies, we used the orthologous genes of the datasets MAMMALIA and VERTEBRATA.
Eleven different filtering setups were considered: (1) RAW: without any alteration the MSA, (2) HMM: HmmCleaner with the default scoring matrix (for nt MSA, low similarity segments were detected on the corresponding protein MSAs and then reported), (3) HMM-L: HmmCleaner with the "large" scoring matrix, (4) PREQUAL: PREQUAL with default parameters, (5) BMGE: BMGE in loose settings, (6) TrimAl: TrimAl with gappy-out option, (7) HMM Random: removal of the same number of residues per sequence as HmmCleaner would have done but at random, (8) HMM + BMGE: running BMGE as in 5 after HmmCleaner, (9) HMM + TrimAl: running TrimAl as in 6 after HmmCleaner,  (10)  MIN: removal of the sequences with < 100 residues and (11) HMM + MIN: a combination of running HmmCleaner then removing sequences as in MIN.
Single-gene trees were inferred with RAxML v8  [38]  with the PROTGAMMALGF model for protein MSAs and the GTRGAMMA model for nt MSAs. Frequencies of correctly recovered clades were computed with a custom script comparing the single-gene trees to the topology of Irisarri et al. [30]  for the VERTEBRATA dataset and to a concatenated tree of the 137 most complete aa MSAs inferred with PhyloBayes-MPI  [39]  using the CAT+G model  [40]  for the MAMMALIA dataset. These two topologies are in agreement with existing knowledge of vertebrate relationships, even if ambiguities persist for a few nodes (e. g. , the relative position of Xenarthra and Afrotheria). For each MSA, branch lengths were computed with RAxML using the same model as previously while constraining the topology to the respective reference species tree.
Acknowledgments We thank  Nicolas Lartillot  for his numerous insights throughout the method development, as well as for comments on the manuscript,  Henner Brinkmann  for his help in method validation,  Rik Verdonck  for critical reading, and two anonymous reviewers for helpful comments. Computations were made on the supercomputers Mp2 and Ms2 from the  Université de Sheerbrooke , managed by  Calcul Québec and Compute Canada. The operation of this supercomputer is funded by the  Canada Foundation  for.
Availability of data and materials HmmCleaner is written in Perl 5 and is based on Bio::MUST modules (https:// metacpan. org/author/DBAURAIN), aimed at providing seamless integration with the MUST ecosystem  [41]. It can be downloaded from CPAN at the following address (https://metacpan. org/release/Bio-MUST-Apps-HmmCleaner) or can be directly installed through the various CPAN clients using the module name Bio::MUST::Apps::HmmCleaner. Datasets used in this study are available at the following address: https://doi. org/10. 6084/ m9. figshare. 6004250. v1.
Image acquisition and statistical analysis The gels were scanned with a LabScanner (Amersham Bioscience), and the images analyzed with an ImageMaster 2D Platinum 7. 0 (GE Healthcare) to identify and quantify each spot, considering their area and intensity. Each data spot was transformed into a volume value (volume%) by the software; this volume corresponded to protein accumulation. In addition, for each treatment, a reference gel of the experimental triplicates was established. By comparison between treatments, the software allowed the identification of exclusive spots (i. e. , present only in one treatment) as well as the relative and possibly differential spots between treatments (corresponding to differentially expressed proteins). PCA of differentially expressed proteins was conducted using the "vegan" package in R version 3.