dataesr/scientific-paragraphs-categorization huggingface dataset:
Portal correctly kept the subpopulations belonging to the same type of immune cells close to each other, revealing the resemblance of immune cells across different tissues. For instance, the transcriptional correlation of all types of B cells, containing B cells, naive B cells, immature B cells, precursor B cells, and late pro-B cells confirmed such similarity (Fig. 4c ). In addition, the epithelial cells of different tissues were identified by Portal as disjoint clusters, which was consistent with the biological distinction among these cell types (Fig. 4d ). Portal successfully and efficiently aligns datasets across different data types. As most of existing methods were developed only for integrating scRNA-seq datasets, aligning datasets with different data types could be problematic for these approaches.
Here we illustrate that Portal can flexibly account for the distinction between different data types and yield accurate integration results. We first examined integration of scRNA-seq data and snRNA-seq data. For frozen samples such as biobanked tissues, and for tissue types that have unique morphology or phenotypes, such as brain, fat, or bone, it can be challenging or sometimes even impossible to extract intact cells for scRNA-seq profiling  [45, 46]. To bypass this issue, snRNA-seq has been developed. Although nuclear transcriptomes are shown to be representative of the whole cell  [47] , distinctions between the whole cell and nucleus in terms of the transcript type and composition make scRNA-seq data and snRNA-seq data intrinsically different  [45]. Aligning these two types of data is desirable, as the combined dataset enables joint analysis that can take advantages of both techniques, and help to improve statistical power for the analysis.
Especially for comparing multiple complex tissues, with some cell types being shared and others being non-overlapping, researchers could benefit from such integrated joint analysis -one example being the integration of brain snRNA-seq data with scRNA-seq data of blood to examine similarities and differences between immune cells in each tissue milieu. However, due to the inherent difference in these two data types, aligning scRNA-seq and snRNA-seq data is not the same as batch effects correction. Compared to batch effects among scRNA-seq datasets, technical noise and unwanted variation arising from different data types are often more complex and have higher strength  [45, 48]. Thus, using standard batch effects correction to integrate across data types may result in loss of alignment accuracy or important biological signals.
We evaluated Portal's ability to integrate snRNA-seq data and scRNA-seq data using three mouse brain atlas datasets, including one snRNA-seq dataset profiled by SPLiT-seq  [49] , and two scRNA-seq datasets profiled by Drop-seq and 10X  [8, 9]. In this task, we applied integration methods to harmonize these three atlases across all brain regions. To test the accuracy of integration results, we only used cells that had annotations provided by the authors in each atlas project. After selecting cells with cell type annotations, 319,359 cells in the Drop-seq dataset, 160,678 cells in the 10X dataset, and 74,159 nuclei in the SPLiT-seq remained for integration. Prior to any integration, the raw datasets were clustered by the experimental methods rather than the cell types (Fig. S25a ), and shared cell types between the three datasets did not align well, indicating the initial discrepancy between the three large datasets.
After integration, UMAP visualizations showed that the different alignment methods gave varying results. Portal (Fig. S25b ) and Seurat (Fig. S25d ) achieved the best alignment of data among different methods, showing good mixing of cells annotated with the same cell type label, while also preserving subcluster data structure in the integrated results. In particular, the alignment of scRNA-seq (10X, Drop-seq) and snRNA-seq (SPLiT-seq) datasets was comparably good as that of the two scRNA-seq datasets, indicating successful alignment between the two data types without loss of biologically important variations between clusters. Online iNMF (Fig. S25e ), although it successfully clustered and aligned the same cell types together, within each cluster the streaky pattern suggested potential numerical artefacts in the integrated data.
Furthermore, online iNMF alignment resulted in loss of biological variation, which was most easily observable in the coalescence of the previously distinct neuron subpopulations (Fig. S25a ) into one large amorphous cluster (Fig. S25e ). Harmony, however, showed poor mixing of the snRNA-seq data in some of the cell types, such as the astrocytes, where the scRNA-seq datasets were well-mixed after alignment, but the snRNA-seq data were not mixed well with the rest (Fig. S25c ). Similar to online iNMF, some of the neurons' subcluster structure appeared to be lost after the integration by Harmony. Overall, Portal and Seurat presented the best scRNA-seq and snRNA-seq data alignment performance; however, not including data preprocessing time, Seurat took over 17 hours to complete the task, while Portal only took 87 seconds (details of the procedure are included in the Methods section).
We further assessed Portal's ability to integrate across data types when no cell type, or very few cell types are shared. In this scenario, we applied Portal to integrate one human PBMC scRNA-seq dataset  [50]  with two human brain snRNA-seq datasets  [51, 52] , respectively, as two examples. In the first example (Fig. S26 ), where no cell type was shared between datasets, Portal did not mixed any two populations of cells together, showing its robustness. More importantly, it embedded monocytes and dendritic cells from the PBMC dataset close to macrophages from the brain dataset, indicating the similarities among these immune cells across tissue types. In comparison, overcorrection was observed in results from other state-of-the-art methods. For example, Seurat mixed T cells from blood with excitatory neurons and inhibitory neurons from brain inappropriately (Fig.
S26 ). The reliability of Portal was also demonstrated in the second example (Fig. S27 ). It correctly aligned cells of the only shared cell type (T cell) between datasets, while it did not mix other distinct cell types (Fig. S27 ). Besides integration of scRNA-seq data and snRNA-seq data, we then applied Portal to align scRNA-seq data and scATAC-seq data. As an epigenomic profiling method, scATAC-seq measures chromatin accessibility, providing a complementary view to scRNA-seq. Integrative analyses of scRNA-seq and scATAC-seq data are very helpful to leverage and unify information from the both aspects  [53, 22]. For this task, we used one scRNA-seq PBMC dataset profiled by CITE-seq and one scATAC-seq PBMC dataset profiled by ASAP-seq  [54]. For a better evaluation, we compared Portal with Seurat, online iNMF and VIPCCA, which had shown their ability of cross-omics integration in the original publications.
A recent state-of-the-art method, scJoint  [55] , was also included in the comparison, as it was designed specifically for scRNA-seq and scATAC-seq data alignment. As shown in the UMAP visualizations (Fig. 5 ), Portal, scJoint and VIPCCA were able to align the two datasets correctly, while online iNMF and Seurat did not align some cell clusters: for example, monocytes in online iNMF's integration, and a cluster of mixed cell types from ASAP-seq in Seurat's result. Among the benchmarked methods, Portal showed superior performance on the preservation of biological signals. After Portal's integration, B cells and T cells were kept as disjoint clusters, while subpopulations of T cells were remained to be close to each other. In comparison, the coalescence of the previously distinct cell type clusters in VIPCCA's result indicates the loss of information.
Unlike other methods, scJoint requires cell type label information of scRNA-seq datasets as its input. It utilizes the cell type annotations to construct embedding of cells. As a result, cells from the scRNA-seq dataset with different cell type labels are forced to form disjoint clusters. Biological information was largely lost in scJoint's integration of PBMC data: the subpopulations of T cells (naive CD4+ T cells, naive CD8+ T cells, effector CD4+ T cells, effector CD8+ T cells) lost their similarity and became far apart from each other (Fig. 5 ). Portal and scJoint were also benchmarked with a more challenging task: we manually removed B cells from the CITE-seq dataset such that B cells became a dataset-specific population. The results further demonstrated Portal's robustness to unbalanced cell type compositions even in cross-omics integration, while scJoint showed comparatively inferior performance (Fig.
S28 ). Portal aligns spermatogenesis differentiation process across multiple species. Portal does not need to specify the structure and the strength of unwanted variation when integrating datasets. Instead, it can flexibly account for general difference between datasets, including batch effects, technical noises, and other sources of unwanted variation, by nonlinear encoders and generators in the adversarial domain translation framework. Therefore, Portal is also applicable for merging datasets with intrinsic biological divergence, revealing biologically meaningful connections among these datasets. In this section, we demonstrate that Portal can successfully align scRNA-seq datasets of the testes from different species including mouse, macaque and human (Fig. 6 ). Compared to merging datasets from the same species, cross-species integration poses additional unique challenges.
Although the transcriptomes of different species may share expression of homologous or orthologous genes, the number of shared genes varies between different species and is limited. Furthermore, two species may have genes with very similar sequence and be annotated in the transcriptome by the same name, but have altered function, which means that expression of the same gene in different species can denote different cell function  [56]. In other words, the amount of information one can utilize for integration becomes limited and fuzzier while the variation across datasets becomes far larger, with limited number of shared genes and even fewer shared highly variable genes across different species. Nonetheless, cross-species integration can be very meaningful despite its challenges, as it can generate quick draft annotations of new or less-studied species' atlases and cell types via label transfer from well-studied species.
This evidence suggests that while the evolution of genes on the Y-chr generated diverse species-specific genetic combinations, the tight control of gene expression through MSCI is required to ensure genetic stability  [61]. Recently, cross-species comparisons of "escape genes" that are able to maintain or re-activate their expression despite MSCI repression during spermatogenesis have generated fascinating insights on evolutionary biology, and on sex chromosome evolution  [63, 65, 69, 16]. In this biological context, integrating datasets with continuous and gradient developmental trajectories, such as for spermatogenesis data, requires integration methods to preserve the continuous structure of each dataset, while still providing high accuracy of cell type alignment between datasets. This is more difficult when, like in spermatogenesis data, there are no distinct clusters, making integration of such data a particularly difficult task.
After confirming Portal's capability of preserving the gradual change of cells based on two examples (Figs. S29  and S30 ), we perform cross-species integration of testes datasets from three species, including one mouse  [59] , one macaque and one human  [16] , aligning the different stages of spermatogenesis across species thereby highlighting unique features of each. The successful integration of these spermatogenesis trajectories serves as a demonstration of the power of Portal in complex and low-information data alignment, and how it can facilitate the annotation and discovery process for new single-cell tissue atlases. We first annotated the mouse sample according to the pattern of marker genes (Spermatogonia: Sycp1, Uchl1, Crabp1, Stra8; Spermatocytes: Piwil1, Pttg1, Insl6, Spag6; Early spermatids: Tssk1, Acrv1, Spaca1, Tsga8; Late spermatids: Prm1, Prm2, Tnp1, Tnp2)  [57, 58].
For example, human genes SYCP3, YBX2, SPACA4, H1FNT, PRM1, and TNP1 were known to mark human spermatogenesis progression, but they were absent in the macaque dataset. As only highly variable genes that were expressed in all three species were considered in the integration process, these genes were not used by Portal. However, they showed clear expression in the cell clusters where they were expected to be expressed after integration (Fig. S33 ), confirming the correctness of Portal's integration result. The above results show that Portal can provide an accurate integration even for genes not measured by all three samples. As a comparison, Harmony, Seurat and online iNMF were also applied. However, Harmony and online iNMF were unable to maintain the gradient developmental trajectories of spermatogenesis process for at least one species.
Discussion Taking advantage of machine learning methodologies, Portal is an efficient and powerful tool for single-cell data integration that easily scales to handle large datasets with sample sizes in the millions. As a machine learning-based model, Portal is easy to train, and its training process is greatly accelerated by using GPUs. Meanwhile, mini-batch optimization allows Portal to be trained with a low memory usage. Besides, it also makes Portal applicable in the situation where the dataset is not fully observed, but arrives incrementally. The nonlinearity of neural networks makes Portal a flexible approach that can adjust for complex dataset-specific effects. Nonetheless, according to benchmarking studies, strong ability for removing dataset-specific effects often comes with the weakness in conserving biological variation  [48, 30] , e.
g. , being prone to overcorrection. Portal overcomes this challenge by its model and algorithm designs. First, the boundaries of discriminator scores help Portal to protect dataset-unique cell types from overcorrection. Second, the use of three specifically designed regularizers not only assists Portal to find correct correspondence across domains, but also enables Portal to have high-level preservation of subcluster and small cluster identities in both datasets. Two existing popular methods are Seurat and BBKNN. Seurat often provides integration results with high accuracy, but also requires high computational cost, preventing its usage on large-scale datasets; while BBKNN is well-known for its extremely fast speed, its comparatively less precise results are sometimes a concern for users (Figs.
S5 -S17). A major advance of Portal over these existing state-of-the-art integration approaches is its ability to achieve high efficiency and accuracy simultaneously. With speed comparable or faster than BBKNN, and significantly lower memory requirement than BBKNN (Fig. S18 ), Portal presents similar batch correction performance as well as superior information preservation performance compared to that of Seurat (Figs. S5 -S17 ). Portal also has advantages over several existing deep learning-based methods for singlecell data integration. Currently, the majority of deep learning-based methods leverages the variational autoencoders (VAEs) framework  [70]. scVI  [25] , as a prominent representative of VAE-based methods, is scalable to atlas-level datasets. It utilizes the zero-inflated negative binomial (ZINB) distribution in its modeling, which may be less efficient in capturing complex data structures  [24].
Consequently, the second stage of correcting batch effects heavily relies on the first stage of constructing MNN pairs. Moreover, searching MNN pairs is usually performed on CPUs and could be less computationally efficient for larger datasets. Some deep learning-based methods focus on integrating cross-omics datasets, including cross-modal autoencoder  [75]  and scJoint  [55]. However, they require additional information like cell type information or paired data points for data alignment. They may not be applicable when such information is unavailable. Compared to existing deep learning-based methods, Portal neither relies on a parametric distribution for single-cell data, nor requires MNN pairs to serve as anchors for integration. Owing to its unified framework with unique designs for single-cell datasets, Portal enjoys high flexibility to handle complex datasets and dataset-specific effects with varying strength, and high scalability to deal with millions of cells efficiently.
By leveraging the adversarial domain translation framework, Portal can build meaningful alignment between datasets with efficient utilization of information. From single tissue types to complex cell atlases, Portal showed extraordinary information preservation performance throughout all integration tasks. This feature of Portal is exemplified by integration of the spermatogenesis trajectory across three species, where only a limited number of highly variable genes were shared and utilized by Portal. Improvements can further be made if an effective way of leveraging the whole transcriptome of all species is developed, which is left for future work to address. Nonetheless, such cross-species integration allows biologists to easily identify shared and divergent cellular programs across different species, which is particularly useful for addressing questions of evolutionary biology.
In our example of mouse, macaque, and human testes tissue integration, identifying genes that are primate-specific can help to generate hypotheses about the evolution of primates and shed light on the applicability of various animal models for biological research. It is now clear that using single-cell technologies to assemble comprehensive whole organism atlases encompassing diverse cell types is accelerating biological discovery, and this demand will only grow as more datasets are generated. The demand for integration of such datasets, along with the size of these datasets, will expand correspondingly. We expect that Portal, with its fast, versatile, and robust integration performance, will play a valuable and essential role in the modern life scientist's single-cell analysis.
The key idea is that a cell and its transferred version should not be largely different from each other in terms of cosine similarity. This is because cosine similarity is scale invariant and insensitive to domain-specific effects, including differences in sequencing depth and capture efficiency of protocols used across datasets  [77, 26, 21]. Thus, the cosine similarity regularizer is helpful to uncover robust correspondence between cells of the same cell type across domains. R LA = 1 q {E [∥E 1 (x) -E 2 (G 2 (E 1 (x)))∥ 2 2 ] + E [∥E 2 (y) -E 1 (G 1 (E 2 (y)))∥ 2 2 ]}, q is the dimensionality of Z, E 1 (x) is the latent code of a real cell x ∈ X and E 2 (G 2 (E 1 (x))) is the latent code of its transferred version, E 2 (y) is the latent code of a real cell y ∈ Y and E 1 (G 1 (E 2 (y))) is the latent code of its transferred version.
Analysis details Data preprocessing. We used raw read or unique molecular identifier (UMI) matrices depending on the data source for all scRNA-seq and snRNA-seq datasets, and gene activity matrices for scATAC-seq datasets. We then performed standard data preprocessing for each count matrix, including log-normalization, feature selection, scaling and dimensionality reduction. For each dataset represented by a cell-by-gene count matrix, we first adopted the log-normalization, following the Seurat and Scanpy pipelines  [22, 78]. For each cell, its library size was normalized to 10, 000 reads. Specifically, the counts abundance of each gene was divided by the total counts for each cell, then multiplied by a scaling factor of 10, 000. The normalized dataset was then transformed to log scale by the function log(1 + x).
In order to identify a subset of features that highlight variability across individual cells, we adopted the feature selection procedure from the Seurat pipeline. For each dataset, we selected K top highly variable genes ranked by dispersion with the control of means. In this paper, we used K = 4, 000 throughout all analyses except for the cross-species analysis. In the cross-species analysis, we used K = 3, 000 since the usage of a larger number of features would result in the situation that correspondence across species is dominated by the distinction (e. g. , altered functions of genes annotated by the same name). For each selected variable gene, we centered and standardized its expressions across individual cells to have mean at zero and variance at one. After the above procedures, which were applied to individual datasets, we continued to preprocess data across datasets.
For those datasets to be integrated, we collected genes that were identified as top highly variable genes in all of them as features for integration. We extracted the scaled data with these features from each dataset, and then concatenated them based on features to perform joint PCA. Top p = 30 principle components were kept for all dataset as inputs to Portal. For the shared latent space, we set its dimensionality to be q = 20 throughout all analyses. Unifying gene names for cross-species integration. We retrieved pairwise orthologues (human vs mouse, human vs macaque) respectively from Ensembl Biomart, and merged them to obtain one-to-one-to-one orthologues by using human Ensembl gene names as reference. One-to-one-to-one orthologues across the three species were used to unify gene names.
Genes included in the list were used by Portal. To facilitate the usage of Portal, we have included the used gene lists (orthologues human mouse. txt, orthologues human macaque. txt) as well as the reproducible code for the cross-species integration, among all details for reproducing the experiments throughout our paper at https://github. com/YangLabHKUST/Portal. Hyperparameter setting. Hyperparameters used in Portal are m, t, λ AE , λ LA , λ cos , where m is the batch size used by Portal for mini-batch training; t is the absolute value of boundaries for the logit of discriminator scores (-t < d i (x) < t, i = 1, 2); λ AE , λ LA , λ cos are coefficients for autoencoder consistency regularizer R AE , latent alignment regularizer R LA and cosine similarity regularizer R cos respectively. Throughout all analyses, we set m = 500, t = 5.
0, λ AE = 10. 0, λ LA = 10. 0. Hyperparameter λ cos was tuned within the range [10. 0, 50. 0] with interval 5. 0 according to the mixing metric, where the mixing metric was designed in Seurat to evaluate how well the datasets mixed after integration. The insight into tuning λ cos is as follows: During domain translations, there is a trade-off between preservation of similarity across domains and flexibility of modeling domain differences. Since R cos is designed to preserve the cosine similarity during translations, a higher value of λ cos can enhance the cosine similarity as the cross-domain correspondence, and a lower λ cos allows domain translation networks to deal with remarkable differences between domains. Following this intuition, we empirically find out that λ cos = 10. 0 has a good performance when harmonizing datasets with intrinsic differences, for example, datasets used in cross-species analysis or cross-modal integration (scRNA-seq and scATAC-seq).