dataesr/scientific-paragraphs-categorization huggingface dataset:
This was the procedure used for all the data sets except Synth_T1_S10, Synth_T10_S10, and Synth_T100_S10 where we used a hierarchical Dirichlet first generating for each MAG an expected strain profile for all samples: μ i,g ∼ Dirichlet(α) (2) with α = 1 but then in each sample the strain proportions were generated as: ρ i,s,g ∼ Dirichlet(μ i θ), ( 3 ) so that the mean strain proportions were μ i with an adjustable precision θ. This allowed us to specify a copy number for each genome g in species i in each sample as y i,s ρ g,s. We then generated 150 million paired-end 2x150 bp reads in total across all samples with Illumina HiSeq error distributions using the ART read simulator  [23]. The code for the synthetic community generation is available from https://github. com/ chrisquince/STRONG_Sim.
AD sample collection We obtained ten samples from a farm anaerobic digestion bioreactor across a period of approximately one year. The sampling times, metadata and accession numbers are given in Additional file 1: Table  S5. The reactor was fed on a mixture of slurry, whey and crop residues, and operated between 35-40°C, with mechanical stirring. Biomass samples were taken directly from the AD reactor by the facility operators and shipped in ice-cooled containers to the University of Warwick. Upon receipt, they were stored at 4°C and then sampled into several 1-5mL aliquots within a few days. DNA was usually extracted from these aliquots immediately but some were first stored in a -80°C freezer until subsequent thawing and extraction.
STRONG pipeline STRONG processes co-assembly graph regions for multiple metagenomic datasets in order to simultaneously infer the composition of closely related strains for a particular MAG and their core gene sequences. Here, we provide an overview of STRONG. We start from a series of S related metagenomic samples, e. g. samples of the same (or highly similar) microbial community taken at different time points or from different locations. The Snakemake based pipeline begins with the recovery of metagenome-assembled genomes (MAGs)  [26]. We perform co-assembly of all available data with the metaSPAdes assembler  [33] , and then bin the contigs obtained by composition and coverage profiles across all available samples with CONCOCT  [3]  or alternatively Metabat2  [25]. Each bin is then analyzed for completeness and contamination based on single-copy core genes, and poor quality bins are discarded.
The default criterion is that a MAG requires greater than or equal to 75% of the SCGs in a single copy. While we currently focus on this combination of software tools, in principle we could use any other software or pipeline for MAG recovery, e. g. we could use MEGAHIT as the primary assembler  [29]. For each MAG we then extract the full or partial sequences of the core genes that we further refer to as single-copy core gene (SCG) sequences. The final coassembly graph produced by metaSPAdes cannot be used for strain resolution because, as with other modern assembly pipelines, variants between closely related strains will be removed during the graph simplification process. Instead, we output the initial graph for the final K-mer length used in the (potentially) iterative assembly following processing by a custom executable -spades-gsimplifier based on the SPAdes codebase -to remove likely erroneous edges using a 'mild' coverage threshold and a tip clipping procedure.
We therefore add an extra level of filtering to the BayesPaths algorithm, iteratively running the program for all SCGs, but then filtering those with mean error rates, defined by Eq. 20, that exceed a default of 2. 5 times the median deviation from the median gene error. Filtering on the median deviation is in general a robust strategy for identifying outliers. As a result of this filtering the pipeline only infers strain sequences on a subset of the input SCGs. We have found, however, that the number of SCGs for which strain haplotypes are inferred is sufficient for phylogenetics.
Relevant subgraph extraction Provided with the predicted (partial) gene sequence, T, and the upper bound on the length of the coding sequence, L, defined as 3α Ūn where Ūn is the average length in amino acids of that SCG in the COG database, and α = 1. 5 by default. The procedure for relevant HRAG subgraph extraction involves the following steps. First, the sequence T is split into two halves, T and T , keeping the correct frame (both T and T are forced to divide by 3). T and T are then processed independently. Without loss of generality we describe the processing of T : 1. Identify the path P corresponding to T in the HRAG. We denote its length as L P. 2. Launch a graph search of the stop codons to the right (left) of the rightmost (leftmost) position of T ( T ). The stop codon search is frame aware and is performed by a depth-first search (DFS) on the graph in which each vertex corresponds to a pair of the HRAG position and the partial sequence of the last traversed codon.
Due to the properties of the procedure and the fact that it deals with DBGs, the actual implementation encodes frame state as an integer [0,2] rather than the string of last partially traversed codon. Vertices of this 'state graph' are naturally connected following the HRAG constraints. The search is cut off whenever a vertex with a frame state encoding a stop codon sequence is identified. Several stop codons can be identified within the same HRAG edge sequence in 'different frames', moreover the procedure correctly identifies all stop codons even if the graph contains cycles (although such subgraphs may be ignored in later stages of the pipeline). This codon search procedure was originally implemented for https://github. com/ablab/orf-search and used in  [15]. 3. The 'backward' search of the stop codons 'to the left' is actually implemented as a 'forward' search of the complementary sequences from the complementary position in the graph.
Subgraph simplification While processing SCG subgraphs from a particular MAG we use the available information on the coverage of the MAG in the dataset. In particular, we set up the simplification module to remove tips (a node with either no successors or predecessors) below a certain length (twice the read length) and edges with coverage below a limit that is the larger of 2. 0 or one per cent of the MAG coverage for regular edges or 2. 5 and two per cent of the MAG coverage for short edges. While simplifying a SCG subgraph, edges connecting it to the rest of the assembly graph should be handled with care (in particular, they should be excluded from the set of potential tips). This is because in the BayesPaths algorithm they form potential sources and sinks of the possible haplotype paths.
Availability of data and materials The anaerobic digester time series have been uploaded to the ENA as part of the study PRJEB45779  [19]. STRONG is freely available from https://github. com/chrisquince/STRONG under the MIT License. The BayesPaths algorithm at https:// github. com/chrisquince/BayesPaths and code for synthetic community generation at https://github. com/chrisquince/ STRONG_Sim. The code in the archived version  [34]  was used to generate the benchmarking results in this paper.
Dealing with uncertainty in applications of machine learning to real-life data critically depends on the knowledge of intrinsic dimensionality (ID). A number of methods have been suggested for the purpose of estimating ID, but no standard package to easily apply them one by one or all at once has been implemented in Python. This technical note introduces scikit-dimension, an open-source Python package for intrinsic dimension estimation. scikit-dimension package provides a uniform implementation of most of the known ID estimators based on scikit-learn application programming interface to evaluate global and local intrinsic dimension, as well as generators of synthetic toy and benchmark datasets widespread in the literature. The package is developed with tools assessing the code quality, coverage, unit testing and continuous integration.
Introduction We present scikit-dimension, an open-source Python package for global and local intrinsic dimension (ID) estimation. The package has two main objectives: (i) foster research in ID estimation by providing code to benchmark algorithms and a platform to share algorithms; (ii) democratize the use of ID estimation by provigin user-friendly implementations of algorithms using Scikit-Learn application programming interface (API)  [1]. ID intuitively refers to the minimum number of parameters required to represent a dataset with satisfactory accuracy. The meaning of "accuracy" can be different among various approaches. ID can be more precisely defined to be n if the data lies closely to a n-dimensional manifold embedded in R d with little information loss, which corresponds to the so called "manifold hypothesis"  [2, 3].
ID can be, however, defined without assuming the existence of a data manifold. In this case, data point cloud characteristics (e. g. , linear separability or pattern of covariance) are compared to a model n-dimensional distribution (e. g. , uniformly sampled n-sphere or n-dimensional isotropic Gaussian distribution) and the term "effective dimensionality" is sometimes used as such n which gives most similar characteristics to the one measured in the studied point cloud  [4, 5]. In scikit-dimension, these two notions are not distinguished. The knowledge of ID is important to determine the choice of machine learning algorithm and anticipate the uncertainty of its predictions. The well-known curse of dimensionality, which states that many problems become exponentially difficult in high dimensions, does not depend on the number of features but on the dataset's ID  [6].
More precisely, the effects of dimensionality curse are expected to be manifested when ID ln(M), where M is the number of data points  [7, 8]. Current ID estimators have diverse operating principles (we refer the reader to  [9]  for an overview). Each ID estimator is developed based on a selected feature (such as number of data points in a sphere of fixed radius, linear separability or expected normalized distance to the closest neighbour) which scales with n: therefore, various ID estimation methods provide different ID values. Each dataset can be characterized by a unique dimensionality profile of ID estimations according to different existing methods, which can serve as an important signature for choosing the most appropriate data analysis method. Dimensionality estimators that provide a single ID value for the whole dataset belong to the category of global estimators.
However, datasets can have complex organizations and contain regions with varying dimensionality  [7]. In such a case, they can be explored using local estimators, which estimate ID in local neighborhoods around each point. The neighbourhoods are typically defined by considering the k closest neighbours. Such approaches also allow repurposing global estimators as local estimators. The idea behind local ID estimation is to operate at a scale where the data manifold can be approximated by its tangent space  [10]. In practice, ID is sensitive to scale and choosing neighbourhood size is a trade-off between opposite requirements  [9, 11] : ideally, the neighbourhood should be big relative to the scale of the noise, and contain enough points. At the same time, it should be small enough to be well approximated by a flat and uniform tangent space.
We perform benchmarking of 19 ID estimators on a large collection of real-life and synthetic datasets. Previously, estimators were benchmarked based mainly on artificial datasets representing uniformly sampled manifolds with known ID  [4, 9, 12] , comparing them for the ability to estimate the ID value correctly. Several ID estimators were used on real-life datasets to evaluate the degree of dimensionality curse in a study of various metrics in data space  [13]. Here we benchmark ID estimation methods focusing on their applicability to a wide range of datasets of different origin, configuration and size. We also look at how different ID estimations are correlated, and show how scikit-dimension can be used to derive a consensus measure of data dimensionality, by averaging multiple individual measures.
Related software Related open source software for ID estimation have previously been developed in different languages such as R, MATLAB or C++ and contributed to the development of scikit-dimension. In particular,  [8, 37, 38] , provide extensive collections of ID estimators and datasets for R users, with  [38]  additionally focusing on dimension reduction algorithms. Similar resources can be found for MATLAB users  [39] [40] [41] [42]. Benchmarking many of the methods for ID estimation included in this package was performed in  [13]. Finally there exist several packages implementing standalone algorithms; in particular for Python, we refer the reader to complementary implementations of the GeoMLE, full correlation dimension, GraphDistancesID algorithms  [43] [44] [45]. To our knowledge, scikit-dimension is the first Python implementation of an extensive collection of ID methods.
Benchmarking scikit-dimension on a large collection of datasets In order to demonstrate applicability of scikit-dimension to a wide range of real-life datasets of various configurations and sizes, we performed a large-scale benchmarking of scikit-dimension using the collection of datasets from OpenML repository  [48]. We selected those datasets having at least 1000 observations and 10 features, without missing values. We excluded those datasets which were difficult to fetch either because of their size or an error in the OpenML API. After filtering out repetitive entries, 499 datasets were collected. The number of observations in them varied from 1002 to 9199930 and the number of features varied from 10 to 13196. We focused only on numerical variables, and we subsampled the number of rows in the matrix to maximum 100000.
All dataset features were scaled to unit interval using Min/Max scaler in Python. In addition, we filtered out approximate non-unique columns and rows in the data matrices since some of the ID methods could be affected by the presence of identical (or approximately identical) rows or columns. We added to the collection 18 datasets, containing single-cell transcriptomic measurements, from the CytoTRACE study  [49]  and 4 largest datasets from The Cancer Genome Atlas (TCGA), containing bulk transcriptomic measurements. Therefore, our final collection contained 521 datasets altogether.
The number of features was reduced for these methods when needed, by using PCA-derived coordinates, and the number of observations were reduced by random subsampling. In the Table  1 , we provide the summary of characteristics of the tested methods. In more detail, the following method features have been evaluated (see Figure  2 ). Firstly, we simply looked at the ranges of ID values produced by the methods across all the datasets. These ranges varied significantly between the methods, especially for the linear ones (Figure  2, A ). Secondly, we tested the methods with respect to their ability to successfully compute ID as a positive finite value. It appeared that certain methods (such as MADA and TLE), in a certain number of cases produced a significant fraction of uninterpretable estimates (such as "nan" or negative value), Figure  2, B.
We assume that in most of such cases, the problem with ID estimation is caused by the method implementation, not anticipating certain relatively rare data point configurations, rather than the methodology itself, and a reasonable ID estimate always exists. Therefore, in case of a method implementation returning uninterpretable value, for further analysis, we considered it possible to impute the ID value from the results of application of other methods, see below. Thirdly, for a small number of datasets we performed a test of their sensitivity to the presence of strongly redundant features. For this purpose, we duplicated all features in a matrix and recomputed ID. The resulting sensitivity is the ratio between the ID computed for the larger matrix and the ID computed for the initial matrix, having no duplicated columns.
It appears that despite most of the methods being robust with respect to such matrix duplication, some (such as PCA-based broken stick or the famous Kaiser methods popular in various fields such as biology  [50, 51] ) tend to be very sensitive, Figure  2 ,C, which is compliant with some previous reports  [13]. Some of the datasets in our collection could be combined in homogeneous groups according to their origin, such as the data coming from Quantitative structure-activity relationship (QSAR)-based quantification of a set of chemicals. The size of the QSAR fingerprint for the molecules is the same in all such datasets (1024 features): therefore, we could assume that the estimate of ID should not vary too much across the datasets from the same group. We computed the coefficient of variation of ID estimate across three such dataset groups, which revealed that certain methods tend to provide less stable estimations than the others, Figure  2, D.
Finally, we recorded the computational time needed for each method. We found that the computational time could be estimated with good precision (R 2 > 0. 93 for all ID estimators) using multiplicative model: Time = c × N α obj × N β var , where N obj and N var are number of objects and features in a dataset, correspondingly. Using this model fit for each method, we estimated the time needed to estimate ID for data matrices of four characteristic sizes, Figure  2, E.
scikit-dimension ID estimates metanalysis After application of scikit-dimension, each dataset was characterized by a vector of 19 measurements of intrinsic dimensionality. The resulting matrix of ID values contained 2. 5% missing values which were imputed using the standard IterativeImputer from sklearn Python package. Using the imputed matrix and scaling it to z-scores, we performed principal component analysis (Figure  3, A, B ). The first principal component explained 42. 6% percent of the total variance in ID estimations, with all of the methods having positive and comparable loadings to the first principal component. This justifies the computation of the "consensus" intrinsic dimension measure, which we define here as the mean value of individual ID estimate zscores. Therefore, the mean ID can take negative or positive values, roughly dividing the datasets into "lower-dimensional" and "higher-dimensional" (Figure  3, A, C ).
The consensus ID estimate weakly negatively correlated with with the number of observations (Pearson ρ = -0. 25, p-value=10 -9 ) and positively correlated with the number of features in the dataset (r=0. 44, p-value=10 -25 ). Nevertheless, even for the datasets with similar matrix shapes, the mean ID estimate could be quite different (Figure  3, C ). The second principal component explained 21. 3% of the total variance in ID estimates. The loadings of this component roughly differentiated between PCA-based ID estimates and "non-linear" ID estimation methods, with one exception in the case of the KNN method. We computed the correlation matrix between the results of application of different ID methods (Figure  3 ,D), which also distinguished two large groups of PCA-based and "nonlinear" methods.
Furthemore, non-linear methods were split into the group of methods producing results similar to correlation (fractal) dimension (CorrInt, MADA, MOM, TwoNN, MLE, TLE) and methods based on concentration of measure phenomena (FisherS, ESS, DANCo, MiND_ML). In order to illustrate the relation between the dataset geometry and the intrinsic dimension, we produced a gallery of Uniform Manifold Approximation and Projection (UMAP) dataset visualizations, with indication of the ambient dataset dimension (number of features) and the estimated ID using all methods, Figure  4. One of the conclusions that can be made from this analysis is that UMAP visualization is not insightfull for truely high-dimensional datasets (starting from ID=10 estimated by FisherS method). Also, some datasets having large ambient dimensions, were characterized by low ID, by most of the methods (e.
Conclusions scikit-dimension is the first to our knowledge package implemented in Python, containing implementations of most used estimators of data intrinsic dimensionality. Benchmarking scikit-dimension on a large collection of real-life and synthetic datasets revealed that different estimators of intrinsic dimensionality possess internal consistency and that the ensemble of ID estimators allows us to achieve more robust classification of datasets into low-or high-dimensional. Future releases of scikit-dimension will continuously seek to incorporate new estimators and benchmark datasets introduced in the literature, or new features such as alternative nearest neighbor search for local ID estimates. The package will also include new ID estimators which can be derived using most recent achievements in understanding the properties of high-dimensional data geometry  [56, 57].
Data Set Drug-event associations were extracted from two complementary datasets: the AOP-wiki database (https://aopwiki. org/, as of August 2020)  (23)  and the U. S. Environmental Protection Agency web-based CompTox Chemistry dashboard (https:// FIGURE 1 | Workflow of the procedure to assess AOPs in the drug exposome. A multistep procedure was developed: (1) Data compilation: drug-event associations were extracted from various data sources (solid lines). Several event types were considered, which are molecular initiating events (MIEs) (purple circles), key events (KEs) (light green circles), and adverse outcomes (AOs) (red circles); (2) Bipartite network development: a bipartite drug-event network was created based on the compiled data. To enrich the created network, KE relationships (KERs) (dash lines) extracted from the AOP-wiki database were identified, allowing to add other KEs (green circles); (3) Generation of the monopartite network: if two events shared at least one drug in the bipartite network, a link was created between these two events in the monopartite network.
The transformation of the bipartite network to a monopartite network was done by using the "guilt by association" principle; (4) Network analysis: the loss of information from the bipartite network to the monopartite network was quantified by calculating the "increase of uncertainty" and the "loss of coverage. " comptox. epa. gov/dashboard/, as of March 2021)  (24). The AOPwiki database is a data source developed by the OECD. From this data source, 316 AOPs and 1,393 relevant events (i. e. , MIE, KE, and AO) that are known to be linked to 615 stressors were extracted. All defined AOPs were taken into consideration, even if they are not yet validated by the OECD. The CompTox database contains a wide range of data related to chemical toxicity including, AOP information, human exposure, and other kwnoledge for over 88,300 chemicals.
Direct and putative associations between 22,038 chemicals and 103 AOPs are stored in this database  (25). As both databases provided information related to various stressor types (drugs, pollutants, etc) we matched the drug names and CAS numbers obtained from the DrugBank  (26)  and the DrugCentral databases  (27)  to be able to keep only information related to drugs, without loosing or duplicating knowledge. Consequently, we were able to extract unique drug-event and drug-AOP associations from these two data sources (Comptox and AOP-wiki). To enrich our list of drug-AOP associations, an additional manual curation of the literature using the PubMed database (https://pubmed. ncbi. nlm. nih. gov/) (as of September 2020) was performed. The searches were done using the exact terms of interest for which we were missing links from the previous data integration step.