dataesr/scientific-paragraphs-categorization huggingface dataset:
The ML steps of the HASTEN protocol (Figure  1  Removing the necessity of docking 99% of the entire giga-scale library allows the HASTEN procedure to complete the screening in around 10-14 days for each of the two targets. Additionally, this approach enables a user to shift the computational load associated with a giga-scale screening project from CPUs (most docking tools) to GPUs. Adjusting the failed score or excluding failed compounds from the training data can improve the recall. When docking was unable to produce a docking score, e. g. because all sampled poses were energetically unfavorable, the original HASTEN protocol associated an arbitrary positive docking score with the affected SMILES strings to mimic a positive and therefore unfavorable energy. We started out with such a 'failed score' of +5.
0 for the SurA target and observed excellent recall values. Herein, we define recall as the number of true 100, 1000, and 10 000 top-scoring compounds according to the brute-force docking approach, that had also been selected for docking by HAS-TEN. With SurA and a failed score of +5. 0, we were able to recall around 95%, 90%, and 85% of the top-scoring 100, 1000, and 10 000 compounds, respectively (see Figure  2  for top 1000 and Table  S1 ). Using the same approach with the GAK target, on the other hand, resulted in recalls of only 70%, 67%, and 59% of the top-scoring 100, 1000, and 10 000 compounds, respectively (Figure  2  and Table  S2 ). One major difference between the datasets is the number of failed compounds: While for SurA less than 3% of all compounds in the ERLL library fail to dock (total failed: 42 149 150 compounds), for GAK, 45% are not docked successfully (704 564 272 compounds, compare also the overall distribution of docking scores in Figure  S1 ).
To improve the recall for the GAK dataset, we next attempted adjusting the failed score parameter: Motivated by our hypothesis, we set the failed score to 0. 0 and thus closer to the dataset mean, which improved the recall of the top-scoring 100, 1000, and 10 000 compounds by 9-13% when compared to the failed score of +5. 0 (compare Figure  2  and Table  S2 ). For SurA, we tested the same approach, which, however, did not consistently improve the recall (Table  S1 ). Finally, we assessed the complete removal of all failed compounds from the training data. Notably, the exclusion of all failed compounds resulted in a recall of 94%, 90%, and 84% of the top 100, 1000, and 10 000 true virtual hits for GAK, which is similar to the results achieved initially with SurA (compare Figure  2  and Tables  S1  and S2 ).
Prospects of further speed-up by hyperparameter choice and protocol modification. A key objective of our case study was to investigate the trade-off between maximizing speed and recall in a screening campaign with HASTEN. It is of note that the results presented herein were achieved with Chemprop default hyperparameters, and no hyperparameter optimization was performed. Given the size of our training data, any optimization would have to be done with similarly sized data and thus take a significant amount of additional time for every alternative explored. While it is reasonable to assume that even better models could be achieved by careful optimization of the hyperparameters, the current work confirms that model quality and recall are good when utilizing the default Chemprop settings. Thus, default hyperparameters represent a suitable choice to harness the time-saving potential of ML-boosted docking with HASTEN.
In our test cases, we achieved highly similar RMSE values for validation and test sets (see Figure  S5  and Tables  S5  and S6 ), suggesting good generalizability. It can further be seen that depending on the protocol and target, RMSE values of individual ML iterations converge well before the final iteration 10. Similarly, recall curves (see Figure  2 ) also often show an earlier convergence. Thus, if speed is of the essence, our results underline that HASTEN could, for example, be stopped already at iteration 5, which doubles the speed-up achieved by ML-boosted docking while often sacrificing less than 5% in recall of the top-scoring compounds. The speed gain could likely be increased even more by utilizing smaller training dataset sizes. This is further supported by a recent screening campaign performed by Orion Pharma, combining HASTEN with Glide SP (Standard Precision) to screen a 4.
1 billion compound version of Enamine REAL against an oncology target. In this case, only two 0. 1% iterations were performed and yielded approximately 100 000 high-scoring virtual hits (estimated recall based on the initial random sampling around 0. 5, data not shown). 40  We conclude that good predictive models can be obtained with Chemprop default hyperparameters and major time investment in parameter optimization appears not strictly necessary. Taken together, our data support early stopping as a viable option to achieve even larger screening speedup. Depending on the emphasis of the screening campaign and the desired outcome, HASTEN runs can thus be tweaked to either maximize the speed or instead focus on maximally improving the recall. HASTEN robustly identifies the same compounds irrespective of the initial random selection.
The more robust a scoring protocol, the easier it should be for the ML model to associate a previously unseen compound structure with an appropriate score. We thus hypothesize that equally powerful models to the ones achieved in this work could be generated with smaller training data stemming from more robust docking approaches, as also underlined by previous work with HASTEN. Importantly, this allows the approach to be tailored to the available resources: A user can either generate more training data with a less robust, but faster, docking methodology or rely on fewer docking results for training when utilizing more robust and computationally expensive methods. One should keep in mind that ligand preparation and explicit brute-force docking steps remain the most time-consuming part of the procedure and should be kept to the required minimum.
To support and potentially speed up future screening efforts involving brute-force docking campaigns, we release the entire prepared ERLL library as Glide-compatible, randomized ready-to-use screening databases. Our results indicated additional time-saving potential in strategies such as early stopping, which we found often associated with only a minor drop in the recall of virtual hits. Depending on the available resources, as well as the desired outcome of the screening campaign, we herein outlined possible modifications to the HASTEN protocol for maximum speed-up or maximum recall of top-scoring hits. It is important to note that we herein compare against brute-force docking as our baseline. Docking scores have known limitations in their ability to rank compounds correctly and identify true actives.
As such, the HASTEN protocol is, by definition, most useful when the chosen docking approach successfully enriches in true actives. In our case study, an excellent recall was achieved when using the default parameters of the ML engine Chemprop. It is however possible that both speed and recall could be further improved by performing hyperparameter optimization. While Chemprop provided excellent results in our case study, we acknowledge that it may not always be the ML engine of choice for every screening scenario. With the growth trend of chemical libraries in mind, novel approaches to facilitate giga-scale screening will continue to be developed. Benchmarking such approaches on ultra-large datasets can be challenging since the generation of such reference data is prohibited by time and resource consumption in many settings.
We thus also release the full giga-scale docking results to provide potential benchmarking datasets for future (ML) approaches, that seek to predict docking scores from SMILES. In conclusion, HASTEN represents a robust approach to identify the bulk of the top-scoring virtual hits of a brute-force giga-scale docking campaign by reducing the required docking calculations by 99% (or more). Using HASTEN for ML-boosted docking is thus one viable strategy in a growing arsenal of methods designed to tackle the challenges associated with screening giga-scale libraries in everyday drug discovery.
Acknowledgement The authors would like to thank  Schrödinger LLC  for their generous supply of academic licenses. We acknowledge  CSC  for generous computational resources and thank  Atte Sillanpää  of  CSC  for continued support and many helpful discussions. Thales Kronenberger  is acknowledged for supplying input data for the SurA target and helpful feedback along the way. T. S. , A. P. and I. P. gratefully acknowledge the support of the  Jane and Aatos Erkko  foundation. L. Y. acknowledges the  Academy of Finland  for funding. This work was supported by the  Academy of Finland  grants # 336473  and # 333191  to A. P.
Associated supplementary material The following Supporting Information is available: Supporting Figures  S1-S5. Supporting Tables S1-S6. Summary of utilized Chemprop parameters. The full prepared ready-to-use Enamine REAL lead-like screening library (March 2021) in Schrödinger Phase database format is made available free of charge at https://doi. org/10. 23729/2de314bb-59af-452a-955c-c2ff0c5ea57f. The final docking results for the two targets SurA and GAK are made available as giga-scale benchmarking datasets free of charge at https://doi. org/10. 23729/2170dc9c-4905-43c3-aeee-a574d360737f.
31 The resulting nonredundant DMRs are then aggregated together in a single DMR file. Each detected DMR is then quantified in each sample by calculating the median smoothed methylation level of all CpG sites within this genomic region. 3 ). We first applied a methylation difference cutoff of at least 30% for each of the eight pairwise comparisons (PreCT_1 vs. each of the four PostCT samples and PreCT_2 vs. each of the four PostCT samples). We next used the linear models for microarray data (LIMMA) package and a p-value cutoff of 0. 05 adjusted with the Benjamini and Hochberg method  32  to compare in a single statistical test the two PreCT samples to each of the four PostCT samples.
Gene association We first assembled a unique set of human reference transcripts. To address this issue, Ensembl  33  and RefSeq  34, 35  transcript annotations from the hg38 release of the human genome were downloaded from the University of California Santa Cruz (UCSC) genome browser website  36  on November 11, 2017. Both transcript annotation files (GTF and GFF formats) were subsequently merged into a combined set of nonredundant human reference transcripts with Cuffcompare. 37 sed on this reference transcriptome, we further associated DMRs with their proximal adjacent human genes, that is, less than 10 kbp away. MethGO was also used to analyze the RRBS data at gene-centric levels based on the features described in the human reference transcriptome (promoters, genes, exons, introns, and intergenic regions).
Genes expressed during early embryo development To identify genes expressed during the early embryo development process, we integrated microarray data (Affymetrix Human Genome U133 Plus 2. 0 Array) published by Xie et al. 40  available on the NCBI Gene Expression Omnibus (GEO) repository under the accession number GSE18290. This dataset includes the transcriptome of 6 embryonic stages (18 samples): 1-, 2-, 4-, and 8-cell stages as well as morula and blastocyst. Raw data (cel files) were normalized and background corrected with the RMA method based on the Brainarray custom CDF environment for directly mapping Affymetrix gene to Entrez gene identifiers (hta20_Hs_ENTREZG version 22. 0. 0). 41 Briefly, genes showing a signal ≥background cutoff (median of the normalized dataset, cutoff 5. 04) and a fold-change ≥ 3 in at least one pairwise comparison were selected.
ACKNOWLEDGMENTS In memory of Bernard Jégou. We thank all the technicians and medical staff of CECOS laboratory and Germetheque biobank, Rennes and Toulouse university hospitals, Toulouse. We offer many thanks to the patient without whom this study would not have been possible. We thank the GenOuest bioinformatics facility for hosting the software as well as all members of the Research Institute for Environmental and Occupational Health (IRSET) for stimulating discussions. We also thank the Institut National de la Santé et de la Recherche Médicale (Inserm), the Centre National de la Recherche Scientifique (CNRS), the Université de Rennes 1, and the French School of Public Health (EHESP) for supporting this work.
Importantly, this principle holds not only for genomes, but also for de Bruijn graphs and many k-mer indexes. We reasoned that phylogenetic trees could be embedded into computational schemes in order to group similar data together, as a preprocessing step for boosting local compressibility of data. The well-known Burrows-Wheeler Transform  21  has a similar purpose in a different context and similar ideas have been used for read and alignment compression  22- 25. Other related ideas have previously been used for scaling up metagenomic classification using taxonomic trees  [26] [27] [28] [29]  and search in protein databases  30, 31. At present, the public version of BLAST is frequently used to identify the species of a given sequence by comparing it to exemplars, but it is impossible to align against all sequenced bacteria.
Despite the increasing number of bacterial assemblies available in the NCBI repositories, the searchable fraction of bacteria is exponentially decreasing over time (Fig. 1a ). This limits our ability to study bacteria in the context of their known diversity, as the gene content of different strains can vary substantially, and important hits can be missed due to the database being unrepresentative. Here, we present a solution to the problem of searching vast libraries of microbial genomes: phylogenetic compression, a technique for an evolutionary-guided compression of arbitrarily sized genome collections. We show that the underlying evolutionary structure of microbes can be efficiently approximated and used as a guide for existing compression and indexing tools. Phylogenetic compression can then be applied to collections of assemblies, de Bruijn graphs, and k-mer indexes, and run in parallel for efficient processing.
We implemented phylogenetic compression protocols for assemblies, for de Bruijn graphs, and for kmer indexes in a tool called MiniPhy (Minimization via Phylogenetic compression, https://github. com/karel-brinda/miniphy). To cluster input genomes, MiniPhy builds upon the empirical observation that microbial genomes in public repositories tend to form clusters corresponding to individual species  35  , and species for individual genomes can be identified rapidly via metagenomic classification  36  (Fig. 1b , Methods). As some of the resulting clusters may be too large or too small, and thus unbalancing downstream parallelization, it further redistributes the clustered genomes into sizeand diversity-balanced batches (Methods, Supplementary Fig. 1 ). This batching enables compression and search in a constant time (using one node per batch on a cluster) or linear time (using a single BIGSI methods on search of all plasmids from the EBI database.
7 ), the resulting phylogenetic compression ratio improved to 12. 3× (72. 8 GB) (Supplementary Table  5 ). To better understand the impact of phylogenetic compression across the tree of life, we analyzed the 661k MiniPhy batches of assemblies and COBS indexes, both before and after compression (Supplementary Fig. 8 ). We found that although the top ten species constituted 80% of the genomic content, they occupied less than half of the database space post-compression for both genome representations (Supplementary Fig. 8 ). Conversely, the 'dustbin' batches, which include genomes from sparsely sampled species, expanded to occupy a proportion that was 9. 4× larger in the database post-compression, compared to their precompression proportion, again for both representations (Supplementary Fig. 8 ).
This consistent effect of compression on both assemblies and COBS indexes suggests that phylogenetic compressibility adheres to the same principles, irrespective of the specific genome representation used, with divergent genomes being a major driver of the final size. To demonstrate the practical utility of phylogenetic compression, we used it to implement BLAST-like search across all high-quality pre-2019 bacteria for standard desktop and laptop computers (Phylign, http://github. com/karel-brinda/phylign, Methods). For a given a set of queries, Phylign first identifies for each query those genomes that match best globally across the whole 661k-HQ collection, by proceeding via progressive in-memory decompression and querying of individual phylogenetically compressed COBS 37 k-mer indexes (described above).
Subsequently, Phylign iterates over the phylogenetically compressed genome assemblies (described above) and computes the corresponding full alignments using on-the-fly instances of Minimap 2 41 (Methods). The choice of tools was arbitrary, and other programs or core data structures could readily be used instead. The resulting requirements amount to only 102 GB disk (for the compressed COBS indexes and assemblies: 195 KB/genome, 0. 329 bits/bp, 23. 0 bits/distinct k-mer) (Supplementary Table  6 ) and 12 GB RAM, and Phylign can thus be deployed on most modern laptop and desktop computers. We first evaluated Phylign with 661k-HQ using three different types of queries -resistance genes (the entire ARG-ANNOT database of resistance genes 42 , n=1,856), plasmids (EBI plasmid database, n=2,826), and a nanopore sequencing experiment (n=158,583 reads), with results available within 3.
9, 11, and 4. 3 hours, respectively, on an iMac desktop (Supplementary Table  7 ). Benchmarking against other tools was not possible, as we were unable to find any tool capable of aligning queries to 661k-HQ in a comparable setup. We therefore used the EBI plasmid dataset to compare Phylign to BIGSI with its original database of 448k genomes (which is essentially a subset of 661k-HQ with 1. 43× less genomes)  17. We found that Phylign was over an order of magnitude faster (Fig. 2b , Supplementary Table  7 ); the search required 74. 1 CPU hours and improved performance by a factor of 28. 6× compared to the same BIGSI benchmark with its smaller database (Fig. 2b , Supplementary Table  7 ), while providing the full alignments rather than presence/absence only (Fig. 2b ). To our knowledge, this is the first time that alignment to a collection of a comparable size and diversity has been locally performed.
DISCUSSION It is hard to overstate the impact on bioinformatics of BLAST 2 , which has allowed biologists across the world to simply and rapidly compare their sequence of interest with essentially all known genomes -to the extent that the tool name has become a verb. The web version provided by NCBI/EBI is so standard that it is easy to overlook how representative or complete its database is. However, twenty-four years on, sequencing data is far outstripping BLAST's ability to keep up. Much work has gone into approximate solutions  15  , but full alignment to the complete corpus of bacterial genomes has remained impossible. We have addressed this problem and made significant progress, via phylogenetic compression, a highly efficient general technique using evolutionary history of microbes to improve existing compressive data structures and search algorithms by orders of magnitude.
However, this is the first time all these ideas have been combined together to improve the scalability of search in large genome databases. As with all forms of compression, our ability to reduce data is fundamentally limited by the underlying entropy. For genome collections, this is not introduced solely by the underlying genetic signal, but it is also tightly connected with the sequencing process and our capacity to reconstruct genomes from sequencing reads. The noise in the underlying k-mer histograms (Supplementary Fig. 7 ) suggests that any method for compression or search will have to address noise in the forms of contamination, missing regions, and technological artifacts, with legacy data posing a major challenge for both storage and analysis. Future methods may choose to incorporate stricter filtering, and as our experiments have demonstrated, this not only helps in reducing data volume but also in improving the quality of search outputs.
Estimation of NCBI Assembly database size. The number of bacteria in the NCBI Assembly database  52  (https://www. ncbi. nlm. nih. gov/assembly/) and their compressed size were estimated from the GenBank assembly summary file https://ftp. ncbi. nlm. nih. gov/genomes/genbank/bacteria/assembly_summary. txt (n=1,280,758 records, downloaded on 2022-11-02). The file was sorted according to the 'seq_rel_date' field and then used for calculating the number of published assemblies till a given date, aggregated per month. The total lengths of assemblies for the corresponding time points were estimated using the mean length of a bacterial genome assembly in the 661k collection (3. 90 Mbp) and then converted to the estimated GZip size as previously. Although updates in the assembly_summary. txt file, such as the removal of old contaminated records, may influence the resulting statistics, a manual inspection during a severalmonths-long period showed only a minimal impact of these changes on the old statistics.
Comparison of BLAST NT and NCBI Assembly database sizes (Fig 1a). To compare the sizes of two databases at the same time points, their respective functions were first interpolated in the logarithmic scale using piecewise linear functions from the data extracted above. The resulting interpolations were then used to calculate the estimated proportion of the sizes of NT and the bacteria in the NCBI Assembly database at regular intervals (monthly). Although minor inaccuracies might be present in the calculations (such as variations in the mean bacterial assembly or in the GZip-bits-per-bp conversion across different versions of the databases), these differences do not impact the overall exponential decrease of data searchability.
For microbes, clustering can be accomplished rapidly by metagenomic classification  36  applied to the raw reads or other methods for species identification. Microbial genomes in public repositories form distinct clusters, usually (but not always) corresponding to individual species  35  , and metagenomic classification can assign individual genomes to these respective clusters, defined by the underlying reference database such as NCBI RefSeq  36. This requires only a constant time per dataset and can be fully parallelized, resulting thus in a constant-time clustering if sufficiently many computational nodes are available. The obtained clusters are then reorganized into batches. First, too small clusters are merged, creating a special pseudo-cluster called dustbin, whose purpose is to collect divergent, weekly compressible genomes from sparsely sampled regions of the tree of life.
Subsequently, the clusters that are too largesuch as those corresponding to oversampled human pathogens (e. g. , S. enterica or E. coli) -as well as the dustbin are then divided into smaller batches, to provide guarantees on the maximum required downstream computational resources per one batch. An additional discussion of batching is provided in Supplementary Note 3. Step 2: Inference of a compressive phylogeny (Fig. 1b(ii) ). In this step, the computed batches are equipped with a so-called compressive phylogeny, which is a phylogeny approximating the true underlying phylogenetic signal with sufficient resolution for compression purposes. If accurate inference methods such as RAxML  53  or FastTree 2 54 cannot be applied due to the associated bioinformatics complexity or high resource requirements, phylogenies can be rapidly estimated via lighter approaches such as the Mashtree algorithm  34  (reimplemented more efficiently in Attotree, https://github.