dataesr/scientific-paragraphs-categorization huggingface dataset:
Indeed, with the Capercaillie models important elements such as the tree species or human disturbances are not taken into account  (Coppes, Judith, Dominik, Rudi, & Veronika, 2017; Sachot, Perrin, & Neet, 2003) , contrary to the virtual species case where all the variables influencing the distribution were known. In addition, even if Maxent is known to be robust to sample size effects  (Hernandez, Graham, Master, & Albert, 2006 ), a datasets with fewer than 30 observations can give inconsistent prediction  (Wisz et al. , 2008). The production of overall inaccurate models may limit the effects of the correction methods, and indeed, some observed model performance were poor (AUC < 0. 5). We can also assume that a lower than expected difference in habitat use between sexes is occurring in Capercaillie populations, as we observed high correlation between sex-specific and generic models and low difference between generic and the sum of sex-specific predictions maps, in contrast to our hypothesis H7.
| CON CLUS ION The use of subjective sampling protocols can yield biased SDM results, when working with species where sexes have different habitat use behavior. When using a subjective sampling design, the habitat preferred by one sex can be under-represented in the dataset, introducing bias in the predictions from SDMs. The two spatial bias correction methods tested here were able to correct this effect in a virtual case study, but not in the case of a forest bird, Capercaillie. The effectiveness of spatial correction methods cannot be taken for granted and other potential sources of bias, such as sex differences in detectability, require additional attention. We argue that spatial correction methods can nevertheless be useful when developing SDMs for subgroups (e. g. , sex and age) with distinct behaviors but that the accuracy of the prediction depends on their ability to well represent the spatial observation bias.
DATA AVA I L A B I L I T Y S TAT E M E N T Data and R code are available at the repository: (https ://gitlab. irstea. fr/anouk. glad/Spati al_sampl ing_bias). The Capercaillie observation dataset is available on request only due to the conservation status of this species in the Jura Massif in France. Please send your request to Ms Depraz at groupe-tetras@wanadoo. fr for this dataset. R code to calculate LiDAR metrics is available at (https ://gitlab. irstea. fr/jeanmatth ieu. monne t/lidaR tRee).
Interest in docking technologies has grown parallel to the ever increasing number and diversity of 3D models for macromolecular therapeutic targets. Structure-Based Virtual Screening (SBVS) aims at leveraging these experimental structures to discover the necessary starting points for the drug discovery process. It is now established that Machine Learning (ML) can strongly enhance the predictive accuracy of scoring functions for SBVS by exploiting large datasets from targets, molecules and their associations. However, with greater choice, the question of which ML-based scoring function is the most suitable for prospective use on a given target has gained importance. Here we analyse two approaches to select an existing scoring function for the target along with a third approach consisting in generating a scoring function tailored to the target.
A historical perspective of PM decoys is available elsewhere  [19, 20]. As further discussed elsewhere  [14] , current SBVS benchmarks do not mimic real test sets and hence their ability to anticipate prospective performance must at best be suboptimal. A prime example of this is MUV  [21] , whose test sets were built from High-Throughput Screening (HTS) datasets. In MUV, actives that are not considered to be well embedded in the true inactives (called decoys in that study) are removed. However, by definition, an optimal embedding is the one that is provided by the real test set (the HTS library) and hence MUV is an unrealistic benchmark in this sense. On the other hand, the ability of benchmarks based on PM decoys to mimic real test sets has also been strongly criticised, as explained in the rest of this section.
The fourth and fifth studies proposed DUDE-trained CNN-based SFs using protein-ligand features have also been tested on datasets with property-unmatched decoys obtaining hit rates between 3 and 4 times higher than those from Vina  [26, 29]  (i. e. not only the performance was not poor, but actually excellent compared to this classical SF). More concretely, each of these datasets generated by Riniker & Landrum  [30]  contained 100 diverse actives of a target along with 10,000 decoys (9,800 of these were effectively selected at random, while the remaining 200 were mildly atom-count-matched to an active). Incidentally, one of these SFs  [26]  was built with the same Ragoza et al. 's CNN algorithm that was later employed by Sieg et al. for their experiments  [20] , but surprisingly not even this directly contradictory result was mentioned by the authors.
Selecting a scoring function based on your own evaluation In case the SFs have not been evaluated properly, or not at all, on the target of interest, making your own evaluation of selected SFs may lead to the identification of a predictive SF. This requires retrieving all known actives for the target in relevant databases  [31, 32]  to include in a test set. The other part of the test set will be a much higher proportion of decoys. As discussed in the previous section, employing PM decoys have important shortcomings arising from not capturing the properties of the test sets of interest. In ML, supervised learning must be performed on a training set that resembles the intended test set, otherwise the resulting model will be unlikely to generalise to that test set. This is consequently an established requirement in QSAR too, where models with applicability domains covering test sets of interest are sought after.
It is important to note too that PM decoys are not required either to train or test QSAR models  [35] , despite predicting exactly the same in vitro potency/affinity endpoints as SFs (e. g. K d is predicted by both SFs  [36, 37]  and QSAR models  [38, 39] ). On the other hand, the number of HTS datasets has strongly increased in recent years  [32, 40]. If available for the target of interest, such dataset constitutes an interesting alternative as a test set (containing true instead of assumed inactives, better characterising the chemical diversity of molecules or representing by definition a realistic active-inactive proportion). It is however advisable to curate HTS datasets prior to modelling  [41]. While HTS datasets have been used to test QSAR models  [42, 43] , this is yet to be done with MLbased SFs.
These studies are very encouraging. For example, Liu et al. [43]  cross-validated a range of QSAR models on data from an HTS on a protein-protein interaction target (off-theshelf classical SFs were also tested on the same validation folds, but not ML-based SFs). They found that the QSAR model using RF was the most predictive. All these models were tested on another dataset from a second HTS subsequently carried out by the authors on the same target. The RF model was also found to be the most predictive in this prospective test, outperforming QSAR models based on deep NN algorithms and classical SFs. This shows again that optimal models can be anticipated without any need for PM decoys. Table  1  compiles several freely-available off-the-shelf generic ML-based SFs that could be used on the test set: NNScore  [44, 45] , Δ vina RF 20  [46] , Δ vina XGB  [47] , Convolutional NN  [26] , RF-Score-VS  [27]  or vScreenML  [48].
Resource B & D SF F M Availability DUDEZ Goldilocks  [      https://github. com/HongjianLi/RF-Score MIEC-SVM  [54]      http://wanglab. ucsd. edu/MIEC-SVM ODDT  [55]      https://github. com/oddt/oddt Descriptor Data Bank  [56]      http://www. descriptordb. com.
The latter has been shown  [16]  to outperform classical SFs such as GlideScore-SP at SBVS by a large margin when using PM decoys (this was also true when using AD property-unmatched decoys). This is exciting given that there are many targets for which target-specific SFs could be generated (e. g. 917 single-protein targets in the ChEMBL database with at least 40 ligands per target using an activity threshold of 10 μM  [57] ). Also, selecting training complexes with similar targets and/or ligands as those in the test set have been found to improve performance in this and related docking problems  [27, 58]. Such transfer learning is particularly important for those targets with few or no known ligands yet, where training or even test sets might not be available. The SF with the best leave-target-out crossvalidation  [27]  would be the most promising to tackle these challenging targets.
Furthermore, there are now tools to facilitate the generation and validation of ML-based SFs by providing integrated and well-documented software packages (Table  1 ). For instance, the Descriptor Data Bank (DDB)  [56]  implements a ML toolbox for automatic filtering and analysis of descriptors (features) as well as SF training and prediction. The descriptor filtering module can filter out irrelevant or noisy descriptors to produce a compact subset from the 2,700 descriptors that are initially considered. There is also a standalone version of DDB for download in the form of Python command line programs and a PyMOL plugin, which can be used for high-throughput processing and visualisation of docked complexes. Another suitable software package is the Open Drug Discovery Toolkit (ODDT)  [55].
This is a modular and comprehensive toolkit for use in chemoinformatics and molecular modelling in either Python programming language or its command line interface. ODDT handles ligand molecules in various file formats, generates features for protein-ligand complexes (e. g. BINANA  [51] , PLEC  [59] ) as well as ligands (e. g. USR  [60] , USRCAT  [61] ) and builds SFs using established ML algorithms such as RF or NN. Features and ML algorithms can build ML-based SFs such as RF-Score  [37] , NNScore  [44, 45]  and PLECscore  [59]  on user-supplied data. Multiple metrics to quantify performance and crossvalidations for model assessment are implemented. The trained SF can be saved to a file, shared between users, and subsequently reloaded and used in the command line directly.
Such works highlight the use of ICA in dimensionality reduction, deconvolution, data pre-processing, meta-analysis, and others applied to different data types (transcriptome, methylome, proteome, single-cell data). We particularly focus on the technical aspects of ICA application in omics studies such as using different protocols, determining the optimal number of components, assessing and improving reproducibility of the ICA results, and comparison with other popular matrix factorization techniques. We discuss the emerging ICA applications to the integrative analysis of multi-level omics datasets and introduce a conceptual view on ICA as a tool for defining functional subsystems of a complex biological system and their interactions under various conditions. Our review is accompanied by a Jupyter notebook which illustrates the discussed concepts and provides a practical tool for applying ICA to the analysis of cancer omics datasets.
Introduction Cancer research is one of the most important providers of large-scale molecular profiling data, which help in understanding not only the state of human cells in disease but also shed light on the normal physiological processes measurable and detectable in various kinds of omics datasets. Determining robust and biologically meaningful ways of quantifying cellular and organismal and normal and pathological physiology using high-throughput molecular data remains a major challenge (making biology a quantitative science). Different kinds of biological processes leave characteristic traces at different levels of genome-wide measurements depending on their nature and timescales: some significantly affect transcriptomes, some rather modify DNA methylation programs or mutational spectrum, others are measurable only at the level of proteome and phosphoproteome.
In order to reliably quantify some of these biological mechanisms, one will need to design multi-omics signatures spanning several levels of molecular data descriptions. On top of this, various technical factors interplay with biological ones, frequently in a way which makes it difficult to clearly distinguish both. Rarely does molecular data "speak for themselves": they need to be properly pre-processed, analyzed in the light of mathematical modeling, statistical assumptions, and prior biological knowledge and, finally, should be represented at some pre-defined level of abstraction. In this sense, one of the simplest paradigms of linear mixture of signals plays a pivotal role in the modern molecular data analysis. In this framework, one assumes that a measurable elementary quantity such as expression of a single gene is a result of weighted summation of some latent, and not always directly observable, factor activities which should have associated numerical values.
The first applications of ICA in biology contrasted it to PCA and standard clustering methods and found that the factors determined through ICA are easier to interpret biologically  [1, 2]. This raised an increase in interest of ICA and its applications in various contexts, and, in particular, in cancer biology  [3, 4]. The success of ICA can be connected to the nature of the statistical assumptions which are used to define the method, that match well the underlying high-dimensional distributions of omics datasets. The principles of ICA are briefly introduced in Section 2. 1. Independent component analysis and matrix factorization approaches are standard methods in the rapidly growing arsenal of machine learning methods applied to the molecular biology and medical data. At the same time, remarkable success has recently been achieved in applying deep learning techniques in certain fields of cancer biology such as clinical imaging of various kinds  [5] [6] [7] [8] [9].
deconvolution, and meta-analysis of multiple omics datasets (Figure  1c ). In this paper, we reviewed most of the recent achievements in computational cancer biology research where ICA was used as the main data analysis tool. We also discussed the practices of ICA applications which appeared to be successful in various contexts. This review is accompanied by interactive Jupyter notebook located at https://github. com/sysbio-curie/ICA-in-Cancer-research-review-materials. Figure  1. Independent component analysis (ICA) is a standard tool for reducing the complexity of omics datasets in cancer biology. (a) ICA belongs to the family of matrix factorization methods, approximating a 2D matrix by a product of two much smaller matrices, containing metagenes and metasamples, in the case of omics data.
Brief Introduction into Matrix Factorization Applied to Omics Data Independent component analysis belongs to a family of matrix factorization methods. Each of these methods takes a rectangular matrix  ∈ ℝ of measurements (in sufficiently a large number of observed samples, N, and with number of observed features, m) as an input and approximates it as a sum of products of p pairs of vectors of size N and m. The fundamental equation for all matrix factorization methods states (note that the product of ak and sk vectors gives a one-rank matrix of the same dimension as X): 𝑿 ∑ 𝒂 𝒌 × 𝒔 𝒌 (*) (1) and the problem of matrix factorization is to find a set of ak and sk such that: Over the last decade, significant experience in applying ICA to different kinds of omics data for addressing various problems has been obtained, including data pre-processing, task of cell type deconvolution, and meta-analysis of multiple omics datasets (Figure  1c ).
In this paper, we reviewed most of the recent achievements in computational cancer biology research where ICA was used as the main data analysis tool. We also discussed the practices of ICA applications which appeared to be successful in various contexts. 𝑿 -∑ 𝒂 𝒌 × 𝒔 𝒌 𝟐 → 𝒎𝒊𝒏 (**) (2) This review is accompanied by interactive Jupyter notebook located at https://github. com/sysbiocurie/ICA-in-Cancer-research-review-materials.
In the geometrical language, the Mahalanobis metrics are introduced into the data space instead of the usual Euclidean. Therefore, after whitening, the covariance matrix of the reduced dataset becomes the identity matrix and PCA becomes inapplicable, since all Gaussian signals have been erased from the data. This makes the use of higher-order moments for finding a rotation of the orthonormal coordinate basis easier, which would maximize the non-Gaussianity of the data point projection distributions along each axis. After such a rotation, in the whitened space, the vectors corresponding to the new axes remain orthogonal while in the original data space they can be strongly correlated (see Figure  2a ). Because of the use of whitening as a preprocessing procedure, ICA is frequently considered as a step on top of PCA, consisting in rotating the coordinate system, by exploiting the information contained in higher than second moments of the multivariate data distribution (Figure  1c ).
Various flavors of ICA have been suggested and some of them were tried on omics data. Bayesian ICA with prior constraints have been suggested and tried on the metabolomics data  [24]. The prior constraints can be non-negativity of the a k and s k vector elements. This allows combining the nice properties of non-negative mixture problem and the requirement for mutual independence of the components. A kernel version of ICA have been developed  [25]  and sparse ICA was proposed in Reference  [26] , but both have not yet found wide applications in omics data analysis (though kernel ICA was exploited in Reference  [27] ). Finally, tensorial ICA was recently developed in References  [28, 29]  and recently applied to the joint analysis of gene expression, copy number changes, and DNA methylation data from colon cancer with some promising results (see more in Section 3.
5). Some flavors of ICA seems interesting to explore more in biological applications, in the view of the concept of the integration of functional subsystems (see Section 3. 6), such as tree-dependent component analysis (TCA)  [30]. This variant of ICA allows clustering of the components such that they remain independent between the clusters and dependent within them. It was tested on fMRI data  [31] , but not yet on large-scale omics datasets.
Various Ways to Apply ICA to Omics Data Besides the choice of ICA algorithm (which is frequently fastICA), there are several choices to be made when ICA is applied to omics datasets. The first evident but non-trivial choice concerns a necessity for data log-transformation, which is especially important in the case of gene expression and protein expression data. On one hand, it is strongly desirable in the case of, for example, RNA-Seq data, since empirically they are found to be characterized by log-normal distribution. When ICA is applied to non-transformed data, the resulting components are frequently dominated by single genes or single samples (e. g. , each sample acts as an independent component), which contradicts the initial concept of linear mixture (nothing or almost nothing is mixed in this case).
Another choice in applying ICA to a matrix of omics measurements is the choice between what distribution independence (or non-Gaussianity) is maximized  [18]. One can maximize the independence of metagenes (vectors s k ) or metasamples (vectors a k ). Technically, the first case corresponds to the application of ICA algorithm to the initial matrix X containing samples as rows and omics variables as columns, and the second case corresponds to the application of ICA to the transposed matrix X. Surprisingly, both ways of applying ICA to omics data are wide-spread, and sometimes it makes an effort to figure out in which way ICA was applied. Some studies aim at maximizing the non-Gaussianity of metagenes  [2, [33] [34] [35] , while others maximize non-Gaussianity of metasamples  [36, 37]. Empirically it was shown that maximizing the non-Gaussianity of metagenes is clearly preferable in gene expression analyses to maximizing the non-Gaussianity of metasamples  [38].
This choice leads to much better reproducibility of metagenes in independent datasets as well as to better interpretability of the components computed within the same dataset. Furthermore, in several studies it was found that stabilized or consensus independent components have better characteristics in terms of generalization and interpretation  [34, [38] [39] [40] [41]. By stabilization one usually means re-computing ICA using multiple random initialization with subsequent clustering of the resulting components  [40, 41]. Alternatively, stabilization can be performed through sub-sampling, i. e. , computing ICA multiple times after removing a certain percentage of samples. Applying stabilization can characterize computed independent components in terms of their stability that can be further used for ranking them.
Each component represented as a metagene was correlated to either immune infiltration-related or proliferation-related meta-metagenes derived from Reference  [33]. This analysis shows that only one of the components was strongly correlated to the cell-cycle, while several can be associated with the presence of an immune-infiltrated ICA-derived signature (this, probably, signifies the ability of ICA to deconvolute the major immune cell types in an unsupervised manner (see, Reference  [42] ); (c) correlations matrix between the metagenes of independent components extracted from the TCGA and METABRIC separately. It shows that, for some components computed for different datasets, there exists a strong and unique association between them, indicating the high reproducibility of the ICA results (e.
g. , see Reference  [38] ). Lastly, in some applications of ICA (e. g. , cell-type deconvolution), it is desirable to fix the orientation of the independent components. We remind that in PCA and ICA, the signs of the elements in the vectors a k and s k can be inverted simultaneously without changing the definition of the component. Some methods (such as BIODICA or DeconICA) avoid this ambiguity by assuming that the heaviest tail of the s k distribution should correspond to positive values, which usually gives satisfactory results. In Reference  [43] , each ICA component was characterized by two sets of top contributing genes, from the negative and the positive side of the metagene weight distribution. The largest such set was called a dominating module and the final orientation of the component was chosen to make the weights of the dominating module positive.
Assessment and Comparison with Other Matrix Factorization Methods In several recent studies, ICA was systematically compared with the other most used matrix factorization methods such as PCA and NMF, using large collections of cancer omics measurements. In Reference  [38] , it was tested which matrix factorization method could produce the most reproducible (i. e. , generalizable) definitions of metagenes. In order to achieve this, a notion of a reciprocal best hit (RBH) graph was borrowed from evolutionary bioinformatics. Reciprocal best hit between two metagenes in two ICA decompositions of different datasets defined "orthologous" metagenes. Several criteria have been used in order to evaluate the modular structure of the RBH graphs resulting from application of various ways of applying ICA, PCA, and NMF to the transcriptomic data.