dataesr/scientific-paragraphs-categorization huggingface dataset:
Background Due to the advent of high-throughput technologies, high-dimensional "omics" data are produced at an increasing pace. In cancer biology, in particular, national and international consortia, such as The Cancer Genome Atlas (TCGA), have profiled thousands of tumor samples for multiple molecular assays, including mRNA, microRNAs, DNA methylation and proteomics  1. Moreover, multi-omics approaches are currently being transposed at single-cell level, which further stresses the need for methods and tools enabling the joint analysis of such large and diverse datasets  2. While multi-omics data are becoming more accessible, and studies combining different omics more frequent, the genuine joint analysis of multi-omics data remain very rare. Achieving proper multi-omics integration is crucial to bridge the gap between the vast amount of available omics and our current understanding of biology.
Designing theoretical and computational approaches for the joint analysis (aka intermediate integration) of multi-omics datasets is currently one of the most relevant and challenging questions in computational biology  3, 4. Indeed, the different types of omics have a large number of heterogeneous biological variables and a relatively low number of biological samples, thereby opening to all the challenges typical of "Big Data". In addition, each omics has its own technological limits, noise, and range of variability. All these elements can mask the underlying biological signals. Multi-omics integrative approaches should be able to capture not only signals shared by all omics data but also those emerging from the complementarity of the various omics data. The joint analysis of multiple omics can be performed with various integrative approaches, classified in broad categories  4, 5.
Here, we report an in-depth comparison of representative state-of-the-art multi-omics joint Dimension Reduction (jDR) approaches, in the context of cancer data analysis. We extensively benchmarked nine approaches, spanning the main mathematical formulations of multi-omics jDR, in three different contexts. First, we simulated multi-omics datasets and evaluated the performance of the nine jDR approaches in retrieving ground-truth sample clustering. Second, we used TCGA multi-omics cancer data to assess the strengths of jDR methods in predicting survival, clinical annotations, and known pathways/biological processes. Finally, we evaluated the performance of the methods in classifying multi-omics single-cell data from cancer cell lines. All these analyses allow formulating recommendations and guidelines for users, as well as indications for methodological improvements for developers.
These methods are based on different underlying mathematical formulations, including Principal Components Analysis, Factor analysis, co-inertia analysis, Gaussian latent model, matrix tri-factorization, Non-negative Matrix Factorization, CCA or tensor representations (Supplementary Table  1 ). We selected nine jDR approaches representative of each of these main mathematical formulations (Table  1 ), focusing on methods able to combine more than two omics, implemented in R or Python, and with software readily available and documented. These jDR approaches are iCluster  15  , Integrative NMF (intNMF)  16  , Joint and Individual Variation Explained (JIVE)  17  , Multiple co-inertia analysis (MCIA)  18  , Multi-Omics Factor Analysis (MOFA)  19  , Multi-Study Factor Analysis (MSFA)  20  , Regularized Generalized Canonical Correlation Analysis (RGCCA)  21  , matrix-tri-factorization (scikit-fusion)  22  and tensorial Independent Component Analysis (tICA)  23.
Some of the selected nine jDR approaches are extensions of DR methods initially designed for single omics datasets: intNMF is an extension of non-Negative Matrix Factorization (NMF); tICA is an extension of Independent Component Analysis (ICA); MCIA and JIVE are different extensions of Principal Component Analysis (PCA); and MOFA, MSFA, and iCluster are extensions of Factor Analysis. As a consequence, the different jDR algorithms make different assumptions on the distribution of the factors (Methods). The different jDR approaches also make different assumptions on the across-omics constraints on the factors. Some algorithms, such as MOFA, consider the factors to be shared across all omics datasets. In contrast, the factors of RGCCA and MCIA are different for each omics layer, i. e. , they are omics-specific factors.
These omics-specific approaches still maximize some measures of interrelation between the omics-specific factors such as their correlation (RGCCA), or their co-inertia (MCIA). Finally, JIVE and MSFA consider mixed factors, decomposing the omics data as the sum of two factorizations, one containing a unique factor matrix common to all omics, and the second having omics-specific factor matrices. These last two categories of methods, omicsspecific and mixed thus address also the complementarity of the multi-omics. The majority of the jDR approaches can manage different features (e. g. , genes, miRNAs, CpGs…), but require a match between the samples of the different omics datasets (columns of the # $ matrices, see Table  1 ). Some algorithms, such as MOFA, scikit-fusion and JIVE, can also cope with omics matrices having not all samples in common.
This is particularly suitable for multi-omics integration given that missing samples are frequent in data collections, such as in TCGA. For the sake of comparison, we applied here all methods considering only the samples profiled for all omics. Tensorial approaches, represented by tICA, require by definition that all matrices# $ have exactly the same samples and features. Nonetheless, the features of multi-omics data are frequently different (e. g. genes, miRNAs). A possible strategy to have the same features for all omics would be to convert all the features to the same level, e. g. gene symbols. This is sometimes unfeasible: miRNAs cannot be converted to gene symbols, for instance. We applied here another strategy, where we considered for each omics the matrix of correlation-of-correlation between samples (Methods).
2) Benchmarking joint Dimensionality Reduction approaches on simulated omics datasets We first evaluated the jDR approaches on artificial multi-omics datasets. We simulated these datasets using the InterSIM CRAN package  24. This package generates three artificial omics datasets with imposed reference clustering by manipulating TCGA ovarian cancer data. Thereby, it avoids making assumptions on the distribution of the simulated data. We simulated multi-omics data with five, ten, and fifteen clusters. In addition, each set of clusters is simulated in two versions, either with all clusters of the same size, or with clusters of variable random sizes (Methods). We applied the nine jDR methods, requiring the decomposition of multi-omics data into five, ten, and fifteen factors, depending on the simulated datasets.
3) Benchmarking joint Dimensionality Reduction approaches on cancer datasets In the second step, we downloaded TCGA multi-omics data for ten different cancer types  10  (https://portal. gdc. cancer. gov/). These data are composed of three omics layers: gene expression, DNA methylation, and miRNA expression. The number of samples ranges from 170 for Acute Myeloid Leukemia (AML) to 621 for Breast cancer. We applied the nine jDR approaches to each of these cancer multi-omics datasets, jointly decomposing them in ten factors, as in the work of Bismeijer and colleagues  26. Most cancer subtyping approaches indeed revealed ten or fewer clusters of samples (i. e. , subtypes). The Factor Analysis approach MSFA did not converge to any solution and was thereby not further considered. Importantly, we do not have ground-truth for cancer subtyping (i.
e. clustering of cancer samples). We hence compared the performances of the remaining eight jDR approaches regarding their ability to identify factors predictive of survival, as well as factors associated with clinical annotations. We also evaluated the weight matrices resulting from the jDR methods, by assessing their enrichment in known biological pathways and processes. To test the association of the jDR factors with survival, we used the Cox proportional-hazards regression model. We observed first that the number of factors associated with survival depends more on the cancer types than on the jDR algorithm (Figure  3  and Supplementary Figure  1 ). Indeed, for three cancer types (Colon, Lung, and Ovarian), none of the jDR methods was able to identify survival-associated factors. This result is in agreement with previous observations testing the association of multi-omics clusters with survival on the same TCGA data with the log-rank test  10.
RGCCA, MCIA, and MOFA are overall the best-performing algorithms, since they rank among the top three methods in 6/10, 5/10, and 5/10 cancer types, respectively. In contrast, intNMF, scikit-fusion, and tICA are less effective (among the top three methods in only two out of ten cancer types). Finally, we assessed the jDR methods performances in associating factors with biological processes and pathways (Figure  4 ). To achieve this goal, we need to take into account genes (i. e. , weight matrices) and not samples (i. e. , factor matrices). We computed the number of metagenes (corresponding to the rows of the transcriptomics weight matrix) enriched in at least one biological annotation from Reactome, Gene Ontology (GO), and cancer Hallmarks annotation databases (Methods). An optimal jDR method should maximize the number of metagenes enriched in at least one biological annotation, while optimizing also the selectivity (defined as above for clinical annotations and in the Methods).
4) Benchmarking joint Dimensionality Reduction approaches on single-cell datasets Similarly to bulk multi-omics analyses, the joint analysis of single-cell multi-omics is expected to provide tremendous power to untangle the cellular complexity. In addition, jDR approaches could compensate for the strong intrinsic limitations of single-cell multi-omics, such as small number of sequencing reads, systematic noise due to the stochasticity of gene expression at single-cell level, or dropouts. However, the nine jDR algorithms that we are considering (excepted MOFA) have been designed and applied to bulk multi-omics data. It is therefore crucial to evaluate and benchmark the performances of these jDR algorithms for single-cell multi-omics integration. To test the jDR approaches on single-cell datasets, we fetched scRNA-seq and scATAC-seq, simultaneously measuring gene expression and chromatin accessibility on three cancer cell lines (HTC, Hela and K562) for a total of 206 cells, and reported in the study of Liu and colleagues  27.
5) Multi-omics mix (momix) Jupyter notebook To foster the reproducibility of all the results and figures presented in this benchmark study, we implemented the corresponding code in a Jupyter notebook available at https://github. com/ComputationalSystemsBiology/momix-notebook together with the associated Conda environment containing all the required libraries installed. Written in R, this notebook is structured in three main parts corresponding to the three test cases here considered (simulated data, bulk TCGA cancer data and single-cell data). Importantly, this notebook can be easily modified to test the nine jDR algorithms on user-provided datasets. The notebook can also be adjusted to benchmark novel jDR algorithms on our three test cases. Full documentation to achieve these goals is included in the notebook.
Discussion We benchmarked in-depth nine jDR algorithms, representative of multi-omics integration approaches, in the context of cancer data analysis. In contrast to existing comparisons  [10] [11] [12] [13]  , our benchmark not only focuses on the evaluation of the clustering outputs, but also evaluates the biological, clinical, and survival annotations of the factors and metagenes. Existing comparisons also mainly use simulated data; we here consider large datasets of bulk cancer multi-omics as well as single-cell data. When performing clustering on simulated multi-omics datasets, intNMF, intrinsically designed as a clustering algorithm, offered the most promising results. In the same sub-benchmark, MCIA, MOFA, and RGCCA showed the best performance among the set of methods not intrinsically designed for clustering.
In the cancer data sub-benchmark, when we evaluated the associations of the factors with survival or clinical annotations, MCIA, JIVE, MOFA, and RGCCA were the most efficient methods. When assessing the associations of the metagenes with biological annotations, MCIA and tICA were the most efficient. Finally, in the last subbenchmark, when clustering single-cell multi-omics data, MSFA and tICA, as well as MCIA and intNMF, outperformed other approaches. As mentioned earlier, intNMF, representative of the Non-negative Matrix Factorization (NMF) approaches, performs well for the clustering tasks, i. e. , for detecting substantial patterns of variation across the omics datasets. This is observed for both simulated bulk data clustering and single-cell data clustering. Hence, intNMF should be prioritized by researchers focusing on clustering samples.
Developers could take advantage of the momix Jupyter notebook using it to compare novel methods with established ones. MCIA  18  , is an extension of co-inertia analysis (CIA) to more than two omics datasets. MCIA factorizes each omics into omics-specific factors # $ = 1 $ / $ + A $ , for % = 1,. , ", by first applying a dimensionality reduction approach, such as PCA, to each omics matrix # $ separately and then maximizing their co-inertia, i. e. the sum of the squared covariance between scores of each factor: KDL,KM N O O. N P O ∑ QCR F (# T $ U T $ , # $ U $ ) < T=> , with RKD(# $ U $ ) = 1and U $ correspond global is implemented in the R package omicade4 (https://bioconductor. org/packages/release/bioc/html/omicade4. html).
Regularized Generalized Canonical Correlation Analysis (RGCCA) RGCCA  21  is one of the most widely used generalizations of CCA to multi-omics data. Similarly to MCIA, RGCCA factorizes each omics into omics-specific factors: # $ = 1 $ / $ + A $ , for % = 1,. , ". RGCCA maximizes the correlation between the omics-specific factors by finding projection vectors V $ such that the projected data have maximal correlation: KDL,KM $,W XCDD(# $ V $ , # W V W )for all possible couples %, Y = 1,. , ". Solving this optimization problem requires inversion of the covariance matrix. However, omics data usually have a higher number of features than samples, and these matrices are therefore not invertible. RGCCA thus apply regularization to CCA. RGCCA is implemented into the CRAN package RGCCA (https://cran.
iCluster iCluster  15  decomposes each omics into the product of a factor matrix that is shared across all omics, and omics-specific weight matrices: # $ = 1 $ F + A $ , for % = 1,. , ". iCluster solves the equation above by first deriving a likelihood-based formulation of the same equation and then applying Expectation-Maximization (EM). The method assumes that both the error A $ and the factor matrix / are normally distributed. Finally, clusters are obtained from the factor matrix by applying K-means. The algorithm is implemented into the CRAN package iCluster (https://rdrr. io/bioc/iClusterPlus/man/iCluster. html).
Multi-Study Factor Analysis (MSFA) MSFA is a generalization of Factor Analysis (FA), which models the omics matrices # $ as the sum of data-specific and shared factors: # $ = a/ $ + b $ c $ + A $ , for % = 1,. , ". whereA $ has a multivariate normal distribution and the marginal distributions of / $ ,c $ and # $ are multivariate normal. MFSA is implemented in R and available at https://github. com/rdevito/MSFA.
Factor selection for performance comparisons The jDR approaches make different assumptions on the cross-omics constraints of the factors. The various jDR can be thus classified in shared factors, omics-specific factors and mixed factors approach. To use the factor matrices to compare the performances of the various jDR methods, e. g. to cluster the samples based on the factors, we had to select which factor matrix to use for each jDR. Shared factors jDR methods compute a unique factor matrix, which is used in our benchmark. Omics-specific jDR methods compute a factor matrix for each omics dataset. In these cases, we selected the factor matrix associated with transcriptomic data for our benchmark. However, jDRs with omics-specific factors maximize correlation or co-inertia between the various omics-specific factor matrices.
Data simulation The simulated multi-omics datasets have been produced by the InterSIM  24  CRAN package. InterSIM simulates multiple interrelated data types with realistic intra-and inter-relationships based on the DNA methylation, mRNA gene expression, and protein expression from TCGA ovarian cancer data. We generated 100 simulated datasets, with a number of clusters set by the user. We considered five, ten and fifteen clusters in this study. The proportion of samples belonging to each subtype is also set by the user, while we considered here two conditions with equally sized clusters and variable random sizes, respectively.
Introduction Thanks to sequencing technologies, the number of available protein sequences has considerably increased in the past years, but their functional and structural annotation remains a bottleneck. This task is thus classically performed in silico by scoring the alignment of new sequences to well-annotated homologs. One of the best-known method is BLAST  [1] , which performs pairwise sequence alignments. The main tools for homology search use now Profile Hidden Markov Models (pHMMs), which model position-specific composition, insertion and deletion probabilities of families of homologous proteins. Two well-known software packages using pHMMs are widely used today: HMMER  [2]  aligns sequences to pHMMs and HH-suite  [3]  takes it further by aligning pHMMs to pHMMs. Despite their solid performance, pHMMs are innerly limited by their positional nature.
[8]  proposed a more general method, MRFalign, which performs MRF-MRF alignments using probabilities estimated by neural networks from amino acid frequencies and mutual information. Unlike SMURF, MRFalign allows dependencies between all positions and MRFs are built on multiple sequence alignments. MRFalign showed better alignment precision and recall than HHalign and HMMER on a dataset of 60K non-redundant SCOP70 protein pairs with less than 40% identity with respect to reference structural alignments made by DeepAlign  [9] , showing the potential of using long-distance information in protein sequence alignment. Meanwhile, another type of MRF led to a breakthrough in the field of contact prediction  [10] : the Potts model. This model was brought forward by Direct Coupling Analysis  [11] , a statistical method to extract direct correlations from multiple sequence alignments.
In this paper, we fully describe ComPotts and focus on its performances in terms of alignment quality. In the following sections, we explain our choices for Potts model inference and we describe our method for aligning them, which builds on the work of Wohlers, Andonov, Malod-Dognin and Klau  [15, 16, 17]  to propose an Integer Linear Programming formulation for this problem, with an adequate scoring function. We assess the quality of ComPotts' alignments with respect to 59 reference pairwise alignments extracted from sisyphus  [18]  and BaliBase3  [19]  databases. On these first experiments, computation time was tractable and ComPotts found better alignments than its main competitors: BLAST, HHalign (which is HHblits' alignment method) and MRFalign.
Furthermore, since our goal is to align Potts models, we need the inferrence to be geared towards similar models for similar MSAs, which is not what inference methods were initially designed for. In an effort towards inferring canonical Potts models, we chose to use CCMpredPy  [27] , a recent Python-based version of CCMpred  [28]  which, instead of using the standard L 2 regularization prior R(v, w) = λ v v 2 2 +λ w w 2 2 , uses a smarter prior on v: R(v, w) = λ v v -v * 2 2 + λ w w 2 2 where v * obeys exp(v * i (a)) q b=1 exp(v * i (b)) = f i (a) which yields the correct probability model if no columns are coupled, i. e. P(x|v, w) = L i=1 P(x i ). Our intuition is that positional parameters should explain the MSA as much as possible and only necessary couplings should be added.
Alignment quality assessment We compared each resulting alignment with the reference pairwise alignment extracted from the multiple sequence alignment, considering the alignment precision with the Modeler score  [31]  # correctly aligned columns # columns in test alignment and the alignment recall with the TC score  [32]  # correctly aligned columns # columns in ref alignment , computed using Edgar's qscore program  [33]  v2. 1. To compare our results, we ran HHalign v3. 0. 3 to align HMMs built on the MSAs outputted by HHblits, MRFalign v0. 90 to align MRFs it built from the sequences, both with default options, and BLASTp v2. 9. 0+ without E-value cutoff. As a control, we also ran Matt v1. 00 on the corresponding PDB structures. Results are summarized in figure  2. Note that Matt failed to run 3 of the alignments.
1 Alignment ID Database % identity Selected sequences AL00049879 sisyphus 11. 7 1g6gA, 1gxcA, 1lgqA, 1mzkA, 1r21A, uhtA, 1ujxA, 1wlnA, 2cswA, 2fezA, 2g1lA AL00055723 sisyphus 6 1tu1A, 1v2bB BB11022 BB3 11. 3 1au7, 1neq, 1r69.
Outline and New Contributions In Section 2 we outline the particular biological model and symbolic problem that we aim to solve: BioModel 26 of the MAPK network, which can be found as Model 26 in the BioModels Database of  (Li et al. , 2010). In Sections 3 and 4 we describe two independent symbolic attempts to solve the problem. The first in Section 3 is able to identify symbolically the multistationarity regions of a 1-dimensional parameter space with a combination of Virtual Substitution and Cylindrical Algebraic Decomposition in the Redlog package for Reduce. The second in Section 4 goes on to give full semi-algebraic solution formulae with a combination of Real Triangularization and Cylindrical Algebraic Decomposition using the Regular Chains Library for Maple. The solutions were obtained in different computer algebra systems using different fundamental algorithms, but all from the family of methods for real quantifier elimination.
We move on in Section 5 to describe a new pre-processing method for the problems inspired by graph theory and Gaussian elimination. Then in Section 6 we describe how a combination of ideas from all three preceding sections can be combined to provide solutions over a 2-dimensional parameter space. In Section 7 we discuss testing the stability of fixed points. Then in Section 8 we consider an alternative larger model from the MAPK network (Model 28 in the BioModels Database of  (Li et al. , 2010) ). In Section 9 we compare the models and detour to describe a symbolic grid sampling approach to this problem, including a comparison of this to a leading numerical solver. We consider how further progress could be achieved in Section 10, identifying a conjecture for determining where multistationarity for MAPK may occur without the costly calculations described.