dataesr/scientific-paragraphs-categorization huggingface dataset:
INTRODUCTION The recent improvements of full genome sequencing technologies, commonly subsumed under the term NGS (next-generation sequencing), have tremendously increased the sequencing throughput. Within 10 years, it rose from 21 billion base pairs  (Venter et al. , 2001; International Human Genome Sequencing Consortium, 2001)  collected over months to about 400 billion base pairs per day (current throughput of Illumina's HiSeq 4000). The costs for producing one million base pairs could also be reduced from many thousands of dollars to a few cents. As a result of this dramatic development, the number of new data submissions, generated by various biotechnological protocols (ChIP-Seq, RNA-Seq, etc. ), to genomic databases has grown dramatically and is expected to continue to increase faster than the cost and capacity of storage devices can keep up.
Ongoing projects like the 100,000 Genome Project  (Caulfield et al. , 2019)  or the American 1,000,000 Genome Project (All of Us (NIH), 2020) will easily produce data in the range of several petabases. This growth not only challenges the storage infrastructures and the processing pipelines of public databases because of the sheer data throughput but also challenges algorithm engineers to improve the efficiency of sequence analysis pipelines and develop new strategies for compression, data parallelism, and concurrent computing. The main task in analyzing NGS data is to search sequencing reads or short sequence patterns (e. g. , read mapping and variant calling) or analyzing expression profiles in large collections of sequences (i. e. a database). Searching the entirety of such databases mentioned above is usually only possible by searching the metadata or a set of results initially obtained from the experiment.
Searching (approximately) for specific genomic sequence in all the data has not been possible in reasonable computational time. The demand for solutions can be seen by the various attempts toward enabling sequence searches on large databases (see  (Marchet et al. , 2019)  for an overview). While the NIH SRA provides a sequence search functionality, the search is restricted to a limited number of experiments. Full-text indexing data structures, such as the FM-index, are currently unable to mine data of this scale. Wordbased indices, such as those used by internet search engines, are not directly appropriate for edit-distance-based biological sequence searches  (Bingmann et al. , 2019). The sequence-specific solution Ca-BLAST  (Berger and Peng, 2013)  and its variants require an index of known genomes, genes or proteins, and thus cannot search for novel phenomena in raw sequencing files.
This holds also true in the field of mapping-based metagenomic binning and quantitation where the relevant microbial databases grow about as fast as the sequence archives. The NCBI Refseq database of prokaryotic genomes contains about 30 GiB of sequence, still small enough to build an FM-index for the genomes, which takes about 24 hr time and about 50 GiB memory  (Dadi et al. , 2018). However, including the draft genomes into the analysis increases the database to 380 GiB. Building a single search structure like an FM-index for this amount of data is infeasible.
Related work The problem of approximately searching queries in ultra-large databases has recently been addressed by several groups, focusing on different applications, but all using methods based on the k-mer content of the databases. In the field of alignment-free metagenomic analysis, which focuses on k-mer based analysis, the size of the data also becomes slowly prohibitive. For example, Kraken  (Wood and Salzberg, 2014)  needs z 147 GiB RAM for indexing 380 GBases. For analyzing RNA-Seq data, some groups aimed at searching the raw files directly for a set of transcripts (  (Solomon and Kingsford, 2016)  and shortly afterward  (Sun et al. , 2017) ). They propose novel solutions to the problem of searching a transcript of interest in all relevant RNA-Seq experiments. Up until recently, these searches were only based on the sequences itself; the tool REINDEER  (Marchet et al.
, 2020)  is the first approach to also account for the sequence abundances. As a benchmark, all three publications use a dataset of 2; 652 RNA-seq sequencing runs of human blood, breast and brain tissue (a total of z6:5 TiB) in which they search for 214; 293 known transcripts. For a single query their methods need in the range of 2 À 20 minutes, which is a tremendous improvement and a speedup of orders of magnitude compared to previous methods. Although a breakthrough, the methods presented by the groups need 4 and 0:3 À 2 days for processing the above set of 214; 293 queries, respectively. Very recently, this time was improved by the Patro group with the tool Mantis in  (Pandey et al. , 2018)  to 82 min. Moreover, Bradley et al. (Bradley et al. , 2019)  propose a Bloom filter based solution that can index about 170 TiB of (repetitive) raw sequence into an index of 1.
General method Raptor stores a representative transformation of the k-mer content of the database that is divided up into a number of bins, typically a few hundred to a few thousand (see STAR Methods section for details). The term representative indicates that the k-mer content could be transformed by a function which reduces its size and distribution (for example, using winnowing minimizers on the text and its reverse complement or using gapped kmers). In this work, we use ungapped ðw; kÞ-minimizers for computing representative k-mers (see also  Chikhi et al. (2016)). A ðw;kÞ-minimizer is essentially the lexicographically minimal k-mer of all k-mers and their reverse complements in a window of size w. The same transformation is applied to the k-mers of the query (see Figure  1  for an example and details).
Evaluation In the following, we report our computational experiments for Raptor. First, we will use an artificial dataset to discuss the limitations of binning directories, the impact of compression, the impact of the use of ðw;kÞ-minimizers for different window sizes, and the time/space trade-off when using different partition sizes. We will also compare different binning directories with Mantis and COBS using this dataset. Secondly, we will evaluate Raptor using a real data set used by several groups to determine the membership of transcripts in RNA-Seq files  (Pandey et al. , 2018)  and compare Raptor with Mantis  (Pandey et al. , 2018)  and COBS  (Bingmann et al. , 2019). Lastly, we evaluated the use of Raptor in conjunction with the distributed read-mapper DREAM-Yara  (Dadi et al. , 2018).
All experiments were conducted on a Dell PowerEdge T640 with an Intel Xeon Gold 6248 CPU using 32 threads and 1 TiB of main memory. All file I/O for the artificial dataset was performed to and from a memory mapped file system (/dev/shm) to eliminate I/O effects, i. e. , disk caching, from the measurements. Unless otherwise stated, all other I/O was performed to and from hard disk drives.
Datasets We created a random DNA sequence of 4 GiB size and divided it into b bins which would correspond to b different genomes in, e. g. , a metagenomic data set. Using the Mason genome variator  (Holtgrewe, 2010) , we then generated 16 similar genomes in each bin which differ about 1% from each other on average. This could be seen as bins containing the genomes for a very homologous species. The total sequence length is hence 64 GiB, however, containing b groups of highly similar sequences. Finally, we uniformly sampled a total of 2 20 reads of length 100 bp from the genomes and introduced 2 errors in each read to simulate a sequencing experiment. This artificial dataset is very balanced, which is the ideal case for the x-PIBF, since its overall size is dependent on the largest bin. We will discuss this issue in the discussion section.
On this data set, we use ð19; 19Þ-minimizers and ð23; 19Þ-minimizers in conjunction with thresholds derived by the k-mer Lemma or our probabilistic thresholding for determining which bin contains the query. The value k = 19 was chosen to make random occurrences seldom (see STAR Methods for a detailed discussion). In order to evaluate our method on real data, we took the dataset from  (Pandey et al. , 2018)  which consists of 2; 568 RNA-Seq experiments. Similarly to  (Pandey et al. , 2018) , we excluded all experiments that have an average read length below 50bp because reads shorter than that are rarely relevant in practice. Furthermore, this allowed us to test the minimizer approach with a broader window size. This left us with 1; 742 RNA-Seq experiments which have a size of around 6 TiB (gzipped FASTQ files).
All tools were tested on this dataset using k = 20, a value used in the competitors' publications which also results in an effective text ratio near 1 (see Table  8 ). For the evaluation of DREAM-Yara, we downloaded the NCBI RefSeq for both archaea and bacteria as of February 14th 2021 and applied TaxSBP to create a taxonomic clustering of the dataset into 64 and 1024 bins. We split the RefSeq according to the clustering into 64 and 1024 gzipped FASTA files containing the sequence information of the respective clusters. Similarly to the artificial dataset experiment, we sampled 2 20 reads uniformly from the bins. Within a bin, the reads were uniformly sampled from the existing sequences. We sampled reads of length 250, and introduced 2 errors in each read.
Construction times are in MM:SS, search times in SS. ss. RAM represents the memory peak in MiB during the construction and search, respectively. Raptor processes a total of 1,048,576 reads, allowing for up to 2 errors. False positives (FP) are reads originating from bin i assigned to a bin js i, neglecting the fact that the read may match with bin j when allowing for 2 errors. Tools like Mantis and COBS, which use a simple percentage cutoff, would suffer in a similar increase in false negatives if they used minimizers. However, they could use our results to adapt their thresholding. In our dataset, the number of minimizers for each read ranges from 15 to 35 while our thresholds range from 4 to 13. For example, for a query of length 100 with 20 minimizers the threshold is 6, which is lower than 0:41, 20 = 8:2.
Comparison with other tools In the following, we compare Raptor with the state-of-the-art tools Mantis  (Pandey et al. , 2018)  and COBS  (Bingmann et al. , 2019)  using the artificial data set and a real data set of 1; 742 RNA-Seq experiments also used in  (Pandey et al. , 2018)  as described earlier. Note that the computational experiments for the real data set only use one thread for all tools, the same as it was done in the competitors' publications. The effect of parallelization was tested using the artificial data set where we used 32 threads. We built an index over the artificial dataset (separated in 64 and 1; 024 bins) with COBS and Mantis for a kmer size of 19. Afterward, we queried the same reads we have already searched with Raptor using BDs. Both COBS and Mantis consider a transcript found if the amount of k-mers found is more or equal to a given threshold.
Instead of using the default threshold of 80 percent, we determined a threshold according to the standard k-mer counting lemma, which was 53 percent. Moreover, we had to adapt our input for the index construction of Mantis by adding random quality scores to our FASTA files because Mantis only accepts FASTQ files as input. But even with this adaptation, Mantis, or more precisely the helper tool Squeaker, resulted in a segmentation fault for the artificial dataset separated in 64 bins, thus we only present result for Mantis with 1; 024 bins. As can be seen in Table  4 , the construction of COBS and Mantis takes at least three times longer than for Raptor. Furthermore, searching with Raptor only needs a fraction of the space (about 5 À 8 GiB vs. 20 GiB) COBS and Mantis need, while being as accurate.
The most striking difference is the search time. For ð19; 19Þ-minimizers, Raptor needs between 0. 9 s for b = 64 and 1. 6 s for b = 1; 024. This is about 144 times faster than COBS and (for b = 1; 024) about 30 times faster than Mantis. Next, we evaluated the tools on the real dataset. We built an index over the dataset (that means for 1; 742 bins) with COBS, Mantis, and Raptor using a binning directory for a k-mer size of 20 (since this value was used in  (Pandey et al. , 2018) ). For Raptor, we created two versions, one using a binning directory with an IBF with ð20; 20Þ-minimizers and one version using a binning directory with an IBF with ð40; 20Þ-minimizers. Mantis uses a cutoff in order to sort out low-frequency k-mers that probably resulted from sequencing errors  (Pandey et al. , 2018).
Using BDs for read mapping In the context of read mapping, we can use the BD as follows. The database would be the reference genome(s) we want to map our reads against. Assume we have divided them into bins such that similar parts of the genomes are placed within the same bin. In the context of metagenomics analysis, this could be achieved by using a taxonomic tree (see also  (Dadi et al. , 2018) ); alternatively, the sequences could be clustered based on similarity. The sequences in the bins could then be indexed using a compressed suffix array or other suitable indices and the BD can be used to distribute the approximate searches. The biological case study in this paper is an example of such an application.
Possible extensions Currently, Raptor stores all representative k-mers, even if some representative k-mers in the reference dataset are ubiquitous, i. e. they appear in all or almost all, e. g. , 95%, bins. While some approaches, like Mantis, exclude certain k-mers from consideration, one could instead exclude them from the IBF and store them in a small lookup table. Whenever such a k-mer is queried, we can increase the counters on all bins and save the lookups in the IBF. This might reduce the size of the IBF and speed up the search time. While not shown in this paper, the update operation on an IBF was already used in DREAM-Yara  (Dadi et al. , 2018)  and ganon  (Piro et al. , 2020). Adding data is trivial since we just need to set the corresponding bits in the x-PIBF. For removing data from the x-PIBF we need to clear and rebuild the affected bins of the update.
Furthermore, we will enhance the IBF by adding a hierarchical structure, i. e. a multi-level tree structure, which will dramatically reduce the number of bins needed to represent a dataset. The sequences (A) and (B) represent the same sequence without and with an error at position 6 replacing a T with a G, respectively. The sequence in (A) has 3 minimizers, one of which (caca) is destroyed by the error position. Hence, we could assume that a sequence with one error at this position has a count of 2. However, introducing the error by replacing T with G has the effect that the first window now has a different minimizer not covering the error position (ggca) and hence (B) still has a minimizer count of 3. Thus, (A) and (B) would be wrongly deemed not matching with 1 error. used the lexicographically smallest k-mer for clarity.
Binning directories We define a binning directory (BD) for the text collection fT j g divided into b bins B i as a data structure that returns the counts of the representative k-mers in the query multiset IðkÞ for each bin B i. In this work a binning directory uses a set membership data structure, namely the x-PIBF, that returns the bin membership as a (compressed) bitvector which we call the binning bitvector. The BD then combines the binning bitvectors into count vectors. Our Tool Raptor uses (probabilistic) thresholding to determine whether a query is in a bin or not. Implementing a simple version is indeed not difficult. The problems lie in the fact that the effective text size nðkÞ can be very large, i. e. 10 10 to 10 12. For example, the metagenomics data set used in  (Dadi et al. , 2018)  contains about 2,10 10 different 19-mers.
The thermodynamic functions have been completed for the mixing energies and the tool was then used in order to investigate the case of a natural I/S hydrothermal series from the Shinzan geothermal field (Japan). Activity diagrams have been calculated including illite/smectite and phase relations are found to be in agreement with previous mineralogical observations and solution chemical analyses. The I/S series from Shinzan is further investigated through reactive transport modelling by using a site-specific, augmented version of the geochemical database. Illitization through the formation of I/S is predicted over realistic reaction times, consistently with available mineralogical observations.
Previous advances and context Until now, much of the available thermodynamic data for clay minerals have been derived either from solution experiments or from predictive calculations. Numerous experimental studies of equilibration in solution have been conducted since the pioneering work of  Reesman and Keller (1968). Equilibration experiments on illite were repeated by  Kittrick (1984)  and  Aja et al. (1991). The case of smectite was investigated, among others, by  Reesman and Keller (1968) ,  May et al. (1986) , and  Kittrick and Peryea (1988). Essene and Peacor (1995)  criticised the solution experiments and the properties derived from these studies have not been included so far in the usual thermodynamic databases for geochemical modelling. To overcome such issues, calorimetric measurements have been carried out in the last decade, on a variety of clay minerals, following methods which are described in detail by  Gailhanou et al.
Combining the approaches developed by  Blanc et al. (2015) ,  Gailhanou et al. (2017)  and  Vieillard et al. (2019)  would allow the prediction capacities to be tested with respect to solubility data obtained or selected by  Gaboreau et al. (2020). A similar approach was proposed by  Vidal and Dubacq (2009)  and  Dubacq et al. (2010). In our case, parameterization and verification are performed based on thermodynamic properties measured specifically on clay minerals. The model developed by  Vidal and Dubacq (2009)  and  Dubacq et al. (2010)  is based on  Chermak and Rimstidt (1989)  work, for the anhydrous part. This latter work had been improved in a version specifically devoted to clay minerals by  Blanc et al. (2015) , providing estimates more accurate for this specific group of minerals.
IMPLEMENTATION OF THE CALCULATION IN CLAYTHERM The tool ClayTherm was developed in Visual Basic©, under the Excel© interface. It estimates the formation and reaction properties for 7, 10 and 14Å anhydrous minerals and 10Å hydrated phases. To facilitate the implementation in most geochemical databases, the temperature dependence of the equilibrium constant, log10K(T), is calculated for 8 temperatures (0, 25, 60, 100, 150, 200, 250, 300°C), using the Cp constant approximation, and the parameters A, C and D for the function  3 ) are provided. Thermodynamic properties of ions are issued from the Thermochimie database  (Giffaut et al. , 2014)  and are consistent with the Thermoddem database  (Blanc et al. , 2012). The sharing of the cations throughout the different crystallographic sites is calculated automatically, according to the rules previously reported.
Comparison with Blanc et al. (2015) and Vieillard et al. (2019) datasets The minerals used to parameterize the model developed by  Blanc et al. (2015)  (Table  12 ,  Blanc et al. 2015)  were first tested, to check the consistency of the tool developed here: -14Å minerals: (Si, Al) ordered, no deviation -10Å minerals: HDC  Vinograd (1995)  model except for eastonite and siderophyllite (Si, Al), the ordered octahedral layer and for paragonite which is (Si, Al) ordered. A typographical error was detected in  Blanc et al. (2015)  where paragonite S lat should equal 305. 20 instead of 305. 31 J. mol -1. K -1 -7Å minerals: (Si, Al) ordered, no deviation. For the 10Å anhydrous and hydrated end-members calculated in  Blanc et al. (2015)  (Table  14 ) and  Vieillard et al. (2019)  (Table  8 ), the results are identical using the HDC ordering model.
(2015)  and  Vieillard et al. (2019)  confirms that ClayTherm results are correct for most clay minerals. It allows typographical errors to be corrected and the predicted properties of cronstedtite to be improved, with respect to  Blanc et al. (2015). Vieillard et al. (2019)  proposed a first selection of solubility data from the literature and comparison with log10K (298. 15 K) estimates gives an average of ± 2. 5 log10 units. The selection was recently refined by  Gaboreau et al. (2020)  who also reported a new set of experimentally measured solubilities for clay minerals: kaolinite, smectite, illite, vermiculite and chlorite. For the latter, equilibrium was clearly not reached, even after 2 years of equilibration at 40°C. Additionally, they selected a set of solubility experiments, from previous literature.
Comparison with solubility datasets Standard deviations can be calculated from Table 2, for differences reported in columns 9 and 10. Without considering chlorite solubility, the standard deviation is ± 1. 3 or ± 0. 4 log10 units, depending whether or not calorimetry for vermiculite is considered. The average of both standard deviations, i. e. ± 0. 9 log10 units, produces a rather high uncertainty for log10K (298. 15 K) estimates (± 1. 7 log10 units). This is still the most reasonable statement that can be made concerning uncertainties for log10K (298. 15 K) estimates in this work. As illustrated in Figure  2 , ClayTherm predictions are in very good agreement with the solubilities measured by  Gaboreau et al. (2020). The relation is still correct with the equilibrium constants extracted from literature by  Gaboreau et al.
(2020) , provided the set of data from  Reesman (1974)  is discarded (set A in Figure  2 ). The reason for discarding this dataset could lie in the unusually high liquid/solid ratio L/S = 200 used by the authors, almost ten times higher than the ratio L/S=24 used by  Misra and Upchurch (1976)  and  Gaboreau et al. (2020). A high L/S ratio could delay the duration required to reach equilibrium and to overestimate the stability of the mineral. The last point strongly departs from the general tendency, point (3) in Figure  2 , which corresponds to the equilibrium constant extracted by  Huang and Keller (1973)  for the Fithian illite. It is interesting to note that the analysis provided by the authors strongly differs from that provided by  Reesman (1974). Furthermore,  Huang and Keller (1973)  do not explain the source of data for the mineral analyses, which could raise questions especially concerning the constant extracted for the Fithian Illite.
Conceptual model The modelling was based on a 1D column representing the vertical profile of the WS4 borehole. The profile consisted in 100 m of smectite, discretized into 100 cells of 1 m (see Figure  8 ). A linear temperature gradient from 220°C (inlet of the geothermal fluid) to 170°C (outlet of the geothermal fluid) was imposed, based on  Inoue et al. (1992). A flowrate of 0. 05 m year -1 was considered. The I/S compositions and thermodynamic properties are given in Annex 1. Geochemical calculations were performed using the code PhreeqC  (Parkhurst and Appelo, 2013)  and the thermodynamic database Thermoddem  (Blanc et al. , 2012) , enriched with thermodynamic data established here (see Table  A1  in Annex 1).