Semi-supervised Knowledge Transfer Across Multi-omic Single-cell Data

Authors: fan  zhang
tianyu  liu
zihao  chen
xiaojiang  peng
chong  chen
xian-sheng  hua
xiao  luo
hongyu  zhao
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-11 09:18:32

Abstract: Knowledge transfer between multi-omic single-cell data aims to effectively transfer cell types from scRNA-seq data to unannotated scATAC-seq data. Several approaches aim to reduce the heterogeneity of multi-omic data while maintaining the discriminability of cell types with extensive annotated data. However, in reality, the cost of collecting both a large amount of labeled scRNA-seq data and scATAC-seq data is expensive. Therefore, this paper explores a practical yet underexplored problem of knowledge transfer across multi-omic single-cell data under cell type scarcity. To address this problem, we propose a semi-supervised knowledge transfer framework named Dual label scArcity elimiNation with Cross-omic multi-samplE Mixup (DANCE). To overcome the label scarcity in scRNA-seq data, we generate pseudo-labels based on optimal transport and merge them into the labeled scRNA-seq data. Moreover, we adopt a divide-and-conquer strategy which divides the scATAC-seq data into source-like and target-specific data. For source-like samples, we employ consistency regularization with random perturbations while for target-specific samples, we select a few candidate labels and progressively eliminate incorrect cell types from the label set for additional supervision. Next, we generate virtual scRNA-seq samples with multi-sample Mixup based on the class-wise similarity to reduce cell heterogeneity. Extensive experiments on many benchmark datasets suggest the superiority of our DANCE over a series of state-of-the-art methods.

Main paper: 
 introduction: 
 In the realm of biology and medicine, many experimental methods [18, 25, 53, 51, 54, 63] using high-throughput sequencing technologies have emerged and characterize diverse properties of single cells. The predominant techniques are single-cell RNA-sequencing (scRNA-seq) [55, 88, 49] and single-cell ATAC-sequencing (scATAC-seq) [15, 75] for understanding complicated organisms and tissues at the single cell level. Specifically, scATAC-seq is an epigenomic profiling method designed to assess chromatin accessibility and provides an additional layer of information that complements scRNA-seq, enhancing the ability to comprehend epigenetic heterogeneity within complex tissues. However, identifying the cell types of scATAC-seq data is challenging because of the high sparsity, large dimensionality, and increasing scale of scATAC-seq data. Considering the lack of cell type annotations for scATAC-seq data, several methods [38, 36, 72] have been proposed to transfer cell types from fully-labeled scRNA-seq data to scATAC-seq data 2 . Among these methods, scJoint [38] introduces a scalable transfer learning framework that successfully incorporates atlas-scale scRNAseq with scATAC-seq data based on kNN techniques. scBridge [36] integrates multi-omic data heterogeneously by identifying reliable scATAC-seq cells with smaller omic differences in comparison to scRNA-seq cells. scNCL [72] employs prior domain knowledge along with contrastive learning to address the challenge of heterogeneous features from different modalities. Despite significant progress made by the aforementioned approaches, their success relies on a substantial amount of annotated scRNA-seq data. In reality, annotating both scRNA-seq data and scATAC-seq data is expensive and challenging. Therefore, this study explores a practical yet unexplored problem of knowledge transfer across multi-omic single-cell data under label scarcity in both scRNA-seq data and scATAC-seq data. In this study, only a small fraction of scRNA-seq data with cell types annotated, while a significant portion of both scRNA-seq data and scATAC-seq data lack annotations. In contrast to previous works assuming abundant labeled scRNA-seq data, this setting aligns more closely with practical scenarios.Addressing this realistic problem requires the consideration of two crucial aspects. Firstly, how to learn representations with distinct cell type discriminability under label scarcity in both scRNA-seq data and scATAC-seq data? Learning representations with good discriminability relies on a large amount of annotated data, yet collecting labeled data is extremely costly and difficult. Therefore, it poses a significant challenge in situations where both scRNA-seq data and scATAC-seq data lack a substantial amount of labels. Secondly, how to reduce cell heterogeneity between different omic data while maintaining cell type discriminability? Another challenge is how to reduce the heterogeneous gap between different omic data, which is crucial for cell type transfer. Additionally, while reducing cell heterogeneity, there is a high likelihood of compromising cell type discriminability. Even worse, this becomes significantly more challenging under label scarcity.To answer these questions, we propose a semi-supervised knowledge transfer framework termed Dual label scArcity elimiNation with Cross-omic Multi-samplE Mixup (DANCE). Since the class distribution of the whole scRNA-seq data is unattainable, we start by injecting semantic knowledge using labeled source scRNA-seq data, and then generate pseudo-labels based on optimal transport (OT) for unlabeled scRNA-seq data. Afterward, the unlabeled scRNA-seq data, along with pseudo-labels, are incorporated into the labeled set. For unlabeled target scATAC-seq data, due to the heterogeneous gap, we adopt a divide-and-conquer strategy, which separates the whole data into source-like data and target-specific data. For source-like samples, we employ consistency regularization to force the model to make consistent predictions after random perturbations. For target-specific samples, we select ambiguous labels and then filter the incorrect labels, which can guide the optimization in a soft manner. To mitigate the heterogeneous gap, we propose to generate virtual scRNA-seq samples by multi-sample Mixup according to class-wise similarity and then we perform a sample-to-sample alignment strategy. The effectiveness of DANCE is validated by adopting a series of benchmark datasets and conducting comprehensive experiments in comparison to various state-of-the-art approaches.To summarize, the main contribution of this paper includes: (1) New Problem. We study a realistic yet underexplored problem of knowledge transfer across multi-omic single-cell data under label scarcity in both scRNA-seq and scATAC-seq data. This problem holds significant implications for reducing the annotation cost of single-cell data. (2) Novel Methodology. We propose a semi-supervised framework named DANCE for this problem. DANCE introduces OT-based dataset expansion and a divide-and-conquer strategy for dual label scarcity elimination. Additionally, DANCE employs an effective alignment strategy of cross-omic multi-sample Mixup to reduce cell heterogeneity. (3) Comprehensive Experiments. Through extensive experiments with different settings on various benchmark datasets, we demonstrate the superiority of DANCE against many state-of-the-art methods. 
multi-omic single-cell data integration: 
 The integration of multi-omic single-cell data constitutes a fundamental challenge in elucidating biological processes. Multi-omic single-cell data affords a multitude of perspectives on cellular functions, thereby augmenting our understanding of biology. Some approaches [69, 44, 59, 37, scRNA-seq (),-Target-specific Samples Iterative TrainingFigure 1 : An overview of DANCE. We first utilize labeled scRNA-seq samples to inject semantics and expand the scRNA-seq dataset by OT-based pseudo-labeling. Then we employ consistency regularization and ambiguous set learning for source-like and target-specific scATAC-seq samples. In addition, we perform cross-omic multi-sample Mixup to reduce cell heterogeneity. 30, 61, 2] opt for denoising, batch correction, and integrating single-cell data across numerous experiments, encompassing transcriptomic data and scATAC-seq data. Nevertheless, the simple application of these methodologies to multi-omic data integration poses computational challenges and frequently yields unsatisfactory results, given the considerable differences in dimensions and sparsity levels among different modalities. Moving a step closer, methodologies such as scAI [25] and MOFA+ [3] utilize factor analysis and joint clustering to concurrently measure multi-omic data within the same cell. Nevertheless, these methods require pairwise data format, and executing paired measurements presents technical challenges and involves high costs. Therefore, another series of approaches have been proposed to measure multi-omic data from different cells, including manifold alignment [63, 1, 39] , matrix factorization [18, 64, 84] , correlation-based methods [5, 54] , and neural network approaches [38, 36, 72] . Recent approaches often map multi-omic data into low-dimensional spaces through shared auto-encoders [21, 66, 56, 20] , and Graph Neural Networks (GNNs) encoders [65, 60, 40] , which focus on knowledge transfer across multi-omic data. For instance, scBridge [36] maps multi-omic single-cell data into a shared embedding space and mines reliable samples for dataset expansion, thereby merging the scATAC-seq and scRNA-seq data into the same dataset. GLUE [8] combines a knowledge-based graph and adversarial alignment to explore regulatory interaction across various omic layers. scCLIP [68] introduces a contrastive learning approach to integrate multi-omic single-cell data. It aligns the representations of pairwise multi-omic single-cell data without the usage of cell type labels. Despite their significant strides in integrating heterogeneous cells, all these methods overlook a practical challenge in reality, i.e., the difficulty of annotating both scRNA-seq data and scATAC-seq data. Therefore, there is an urgent need for an approach that can effectively alleviate label scarcity in knowledge transfer across multi-omic single-cell data.To tackle the label scarcity issue in reality, semi-supervised learning has been widely studied with various applications, such as image classification [46, 23, 7, 6] , semantic segmentation [48, 14] , and object detection [24, 80] . Recent studies on semi-supervised learning include consistency regularization approaches [46, 52, 67] and pseudo-labeling approaches [6, 7, 23, 79] . Consistency regularization usually first introduces random perturbations from various sources, including data input [67] , network parameters [47] , and deep features [28] . By encouraging the model to make consistent predictions under these perturbations, it can still learn good representations even under resource constraints. Pseudo-labeling typically assigns pseudo-labels to unlabeled samples using a neural network. It often comes with some special techniques, such as dynamic thresholding [78, 33] and curriculum learning [29, 22, 10] . Then those pseudo-labels together with their corresponding samples are added to the dataset for future use. Besides the advanced approaches in computer vision, other related areas such as node classification [13, 62] and reinforcement learning [87] also benefit from semi-supervised learning. However, to the best of our knowledge, there exists no previous work to study the problem of semi-supervised knowledge transfer across multi-omic single-cell data. Therefore, we provide a semi-supervised framework termed DANCE to promote the understanding of semi-supervised learning in the field of bioinformatics. 
problem definition: 
 In our problem, we have two datasets D (s) and D (t) , which denote the source scRNA-seq dataset and the target scATAC-seq dataset, respectively. In the semi-supervised setting, the scRNA-seq dataset D (s) is separated into two parts, the labeled dataset D (s),l and the unlabeled dataset D (s),u , where D (s),l = {(x(s),l i , y (s),l i )} N l i=1 consists of N l scRNA-seq samples x (s),l iand the corresponding labels y (s),l i , andD (s),u = {(x (s),u i )} N u i=1 includes N u unlabeled scRNA-seq samples x (s),u i. The target scATAC-seq dataset with the size N t is denoted asD (t) = {(x (t) i )} N t i=1 , which includes N t scATAC-seq samples x (t)i . We focus on the closed-set setting, thus both scRNA-seq data and scATAC-seq data are assumed to share the same set of C cell types. A shared encoder F (•) is adopted to map both the scRNA-seq samples and scATAC-seq samples into the embedding space. Following the encoder, we utilize a classifier H(•) to convert representations into softmax predictions corresponding to different cell types. The representations and predictions are denoted as z i = F (x i ) and p i = sof tmax(H(z i )), respectively. It should be noted that only a small portion of the scRNAseq data is labeled, and a significant number of both scRNA-seq samples and scATAC-seq samples lack cell type annotations. We aim to transfer the semantic information related to different cell types from D (s) to D (t) , which has two expectations: (1) The cell representations of scRNA-seq data and scATAC-seq data are mapped into a shared common embedding space with the similarity structure preserved. (2) The cell type annotations on scATAC-seq data are accurate after label transfer. 4 The Proposed Approach 
framework overview: 
 This paper investigates the problem of semi-supervised knowledge transfer across multi-omic singlecell data. We employ a shared encoder and classifier for heterogeneous scRNA-seq data and scATACseq data. The labeled scRNA-seq data are utilized to inject the semantic knowledge. To overcome label scarcity in source scRNA-seq data, we generate pseudo labels based on optimal transport and merge them into the labeled set. To address label scarcity in target scATAC-seq data, we perform consistency regularization for source-like samples and learn from ambiguous labels for target-specific samples. To reduce the cell heterogeneity, we generate virtual scRNA-seq samples through cross-omic multi-sample Mixup and align the heterogeneous multi-omic data. The overview of the proposed framework is illustrated in Figure 1 , and each component will be elaborated in the next sections.The first obstacle is the label scarcity in scRNA-seq data. Recently, pseudo-labeling is a commonly used technique in semi-supervised learning [52, 78] and domain adaptation [83, 31] problems, which usually operate at the sample level [52, 83] by assigning the label with the highest confidence to each sample. However, this sample-level prediction-based pseudo-labeling strategy could be biased to easy classes, leading to biased and suboptimal results. To tackle this, we incorporate optimal transport (OT) [12, 9, 11] into the pseudo-labeling process, which is capable of taking into account prior cell type distributions from a global view to reduce potential noise in pseudo-labeling.Here we first recall the knowledge of OT. OT is a constrained optimization problem designed to discover the optimal coupling matrix Q for mapping one probability distribution to another, with the goal of minimizing the overall cost. To represent the cost of transporting data from γ and η, we introduce a cost matrix C ∈ R |γ|×|η| , where |γ| and |η| denote the dimension of γ and η, respectively. By applying the Sinkhorn algorithm [17] to minimize the Sinkhorn distance, we can obtain the optimal coupling matrix Q:EQUATIONwhere(γ, η) = Q ∈ R |γ|×|η| + | Q1 |η| = γ, Q ⊤ 1 |γ| = ηand σ is set to 10 empirically.To employ OT for pseudo-labeling, we begin by training the encoder and classifier on labeled scRNAseq data, enabling the injection of semantic knowledge into the model. Similar to traditional cell type annotation problems [16, 77] , we leverage a small number of labeled scRNA-seq data and minimize the cross-entropy between the classifier's output and the cell type labels:EQUATIONwhere p (s),l i represents the predicted distribution of x (s),l i.Then, we aim to transfer the semantic information from labeled scRNA-seq samples to unlabeled ones. In particular, a mini-batch of B unlabeled samples is fed to our neural network to generate the prediction matrix P ∈ R B×C , which could be biased to easy classes and thus inaccurate. To tackle this, we learn an optimal coupling matrix that should obey prior cell type distributions from a global view. Since we do not have prior information, we adopt a uniform distribution as in the polytope( 1 B 1 B , 1 C 1 C ), where 1 B and 1 C indicate two vector of ones with dimensions of B and C, respectively. In addition, the optimal Q should be close to P . Therefore, we define the cost matrix C as -logP , and Eqn. 1 can be reformulated as:EQUATIONThe coupling matrix Q obtained by optimizing Eqn. 3 can be used to generate pseudo-labels for unlabeled scRNA-seq data. Formally, the pseudo-labels can be defined as:EQUATION) Then the unlabeled scRNA-seq samples and their corresponding pseudo-labels would be added to the labeled scRNA-seq dataset:EQUATIONLeveraging OT-based pseudo-labeling for dataset expansion can effectively mitigate label scarcity in scRNA-seq data. On the one hand, we employ labeled scRNA-seq data for semantic injection using Eqn. 2, obtaining high-quality pseudo-labels. On the other hand, with the dataset expanded during the current iteration, more information is available for Eqn. 2 in the subsequent iteration. This leads to generating more reliable pseudo-labels in the following iterations. Through continuous iterative training, we can progressively enhance both the scale and quality of the scRNA-seq dataset. Next, we theoretically support the superiority of OT-based dataset expansion. See proof in Appendix A. Theorem 4.1. Let P n = 1 B P be the normalized version of P . Then after our optimal transport-based dataset expansion, we have (1) For any ε > 0 and large enough σ, we have CE(P n , P ) + ε ≥ CE(Q, P ) + C P , whereC P = C j=1 1 B B i=1 p ij -1 C • 1 B B i=1 -log p ij and CE(Q, P ) = i,j CE(Q ij , P ij ).(2) Let L = (l ij ) B×C denote the soft-version of true cell labels, where l ij = max 1 {cell i belongs to class j} , δ and δ > 0. Suppose 0< C min ≤ l ij /P ij ≤ C max for all 1 ≤ i ≤ B and 1 ≤ j ≤ C. Then, CE(P n , L) + ε + C L ≥ CE(Q, L) + C P , where C L = log C max -log C min .Remarks. C P characterizes how far P n is from( 1 B 1 B , 1 C 1 C ).If the mini-batch was uniformly sampled across cell-types, then -log p ij will not be too large. Hence, on the whole, C P can be relatively small, which demonstrates that our optimization has the potential to achieve the desired distribution.Through OT-based pseudo-labeling, we effectively address the label scarcity issue in scRNA-seq data. However, due to the heterogeneous gap, it would not be optimal when employing a similar approach for scATAC-seq data. Here, to tackle label scarcity in scATAC-seq data, we adopt a divideand-conquer learning strategy, which first divides the scATAC-seq data into source-like data and target-specific data and then utilizes separate approaches for optimization.In particular, by comparing the confidence of the predictions and a threshold, we divide scATAC-seq data into two parts D SL and D T S :EQUATIONwhere p(t) i is the predicted distribution of x (t)i . The threshold τ is used to control the size of two groups and set to 0.9 according to previous works [52, 36] . These source-like samples are more likely to have accurate pseudo-labels with high confidence using the pre-trained model. In contrast, target-specific data are far away from source data distribution and thus the predictions could be noisy. Therefore, we utilize separate strategies for these two parts. Learning from Source-like Data. For these source-like samples, we utilize consistency regularization to provide supervision [4, 32, 50, 52, 85] . Here, we add random perturbations (e.g., Gaussian noise) to data, and then force the network to make consistent predictions for both original and perturbed scATAC-seq samples. In particular, we can use ŷi = argmax(p i ) to obtain the pseudolabel of the original scATAC-seq sample. After that, we can get the prediction pi of the perturbed scATAC-seq sample. Thus, the supervised loss can be formulated as:EQUATIONwhere |D SL | represents the number of unlabeled scATAC-seq samples in D SL . In this way, we combine pseudo-labeling with consistency learning to learn from source-like scATAC-seq samples.Learning from Target-specific scATAC-seq Data. However, due to cell heterogeneity between scRNA-seq and scATAC-seq data, many scATAC-seq samples would be far away from the source modality. They are difficult to predict with low confidence scores [71, 85] . To tackle this challenge, we introduce a soft way to learn from their ambiguous label sets instead of pseudo-labels. Here, for each target-specific scATAC-seq sample, we first select the top-k classes with the highest probabilities to make up the ambiguous label set and then introduce a soft supervised loss for reliable guidance.To be specific, we first recall the predicted distribution p i for each sample x i in D T S . The negative cell types are removed with a significant difference from the cell type with the highest probability. In formulation, the removed labels should satisfy the following conditions:EQUATIONwhere the threshold µ is initially set to 1e -3 and gradually decreases during the training process.The left labels make up the candidate label set:EQUATIONThen, we employ a soft supervision loss to provide reliable guidance for target-specific samples:EQUATIONEQUATIONwhere |D T S | denotes the number of target-specific samples. The weight α ij can allow the model to pay more attention to potential correct cell types with high probabilities.Next, we provide the theoretical underpinnings to show that the quality of the soft labels from the candidate label set is higher than that of hard labels. The proof can be found in Appendix A. Z i ̸ = X i ) = β. Under loss L(p, z) = n i=1 z i log p + (n - n i=1 z i ) log(1 -p), consider two estimates of p:(1) p hard = Argmin p L(p, Z).(EQUATION, where t ∈ (0, 1).Then, there exists t 0 ∈ (0, 1) such that for all t ∈ (t 0 , 1), we haveE ∥p t soft -p∥ 2 ≤ E ∥p hard -p∥ 2 .Remarks. In Eqn. 10, if we take p i [1] = p, p i [0] = 1 -p, and α ij = t if j = Z i , then the estimation under Eqn. 10 will degenerate to p t soft .Despite the proposed techniques in addressing the label scarcity problem, reducing the heterogeneous gap between scRNA-seq data and scATAC-seq data remains an additional challenge. To tackle this challenge, previous works [27, 19, 35, 34, 70] propose many strategies for cross-modal alignment. However, these strategies have two major prerequisites. On the one hand, strategies based on centroid learning [27, 19] require accurate label information from both source and target modalities to aggregate samples of the same class. On the other hand, strategies based on cross-modal contrastive learning [35, 34, 82] require pairwise information. Neither of these prerequisites is applicable to our problem, as pairwise information is difficult to acquire in single-cell data, and the majority of annotation information is missing. Towards this end, we propose a sample-to-sample alignment strategy termed cross-omic multi-sample Mixup [81, 58, 86, 73] , which first calculates the class-wise similarity and then mixes the scRNA-seq samples in the hidden space for cross-omic alignment.In particular, we transform the intrinsic distribution of the source scRNA-seq samples to align with scATAC-seq samples in the hidden space. Since we cannot get accurate pairwise information, we calculate the similarity between scRNA-seq samples and scATAC-seq samples and generate virtual samples instead. Note that the weights of the classifier are considered as prototypes that contain representative information of different cell types. We first form a weight matrixW = [w 1 , w 2 , • • • , w C ] T ∈ R C×D, where D denotes the hidden dimension of the embeddings. Then, the class-wise similarity can be determined using cosine distance:EQUATIONwhere | • | represents the L 2 normalization. For a batch of B (s) scRNA-seq samples and a batch of B (t) scATAC-seq samples, we can obtain a similarity matrix S B ∈ R B (s) ×B (t) using Eqn. 12. For the sake of simplicity, we let B (s) = B (t) = B, and the similarity matrix within a mini-batch can be formulated as S B ∈ R B×B . Afterwards, we employ the softmax function to convert S into the weight matrix M for Mixup, which corresponds to the probability of sharing the same semantics:EQUATIONwhere M i collects the weights of B scATAC-seq samples corresponding to the i-th scRNA-seq sample. Then, we can fuse the scRNA-seq samples into a virtual sample through Mixup:EQUATIONAfter getting the fused scRNA-seq samples, we can perform a sample-to-sample alignment strategy.The loss function can be formulated as: EQUATION)In practice, we first warm up the network through labeled scRNA-seq data, and then conduct the training process in an end-to-end manner. The total loss function can be summarized as:EQUATION)where λ is a coefficient and set to 0.1 empirically.The step-by-step training algorithm of our DANCE is summarized in Appendix B. 
experimental settings: 
 Datasets. To validate the effectiveness of the proposed DANCE, we conduct extensive experiments on several benchmark multi-omic single-cell datasets, with a brief introduction as follows: Mouse Atlas Data [74] . The multi-omics data can be accessed from the Tabula Muris mouse data 3 , along with the quantitative gene activity score matrix. In practice, the dataset is divided into multiple subsets based on the various sources of multi-omics data. CITE-ASAP PBMC Data [45] . It is derived from both control and stimulated conditions. CITE-seq encompasses both antibody-derived tag (ADT) matrices and gene expression matrices, while ASAP-seq allows us to access the ADT matrices and the chromatin accessibility matrices simultaneously. In this research, we extract the gene expression matrices of CITE-seq and the chromatin accessibility matrices of ASAP-seq to construct the scRNA-seq and scATAC-seq datasets. The statistics of these datasets can be seen in Appendix D.Baselines. Various approaches are adopted for performance comparisons, including three domain adaptation methods (DAN [42] , CDAN [43] , and MCC [26] ), one semi-supervised learning method (FixMatch [52] ), and three latest multi-omic single-cell data integration methods (scJoint [38] , scNCL [72] , and scBridge [36] ). A brief introduction of these approaches is provided in Appendix E. Performance Comparison. To assess the performance of various knowledge transfer methods, we conduct comprehensive quantitative experiments (Table 1 ) and qualitative experiments (Figure 2 ). From the results, several conclusions can be drawn: Firstly, consistent results from both quantitative and qualitative experiments reveal that DANCE significantly surpasses existing state-of-the-art approaches. We attribute this to the factor that existing methods often only address the issue of label scarcity in the target scATAC-seq data, overlooking the label scarcity issue in the source scRNAseq data. Consequently, when cell type annotations in scRNA-seq data are limited, not only does the discriminability of representations suffer, but also the heterogeneous gap cannot be effectively eliminated. DANCE excels in three main aspects. Firstly, it utilizes OT-based pseudo-labeling to augment the scRNA-seq dataset, effectively alleviating label scarcity in scRNA-seq data. Secondly, for scATAC-seq data, we employ a divide-and-conquer strategy with consistency regularization and ambiguous set learning for two subsets of samples, addressing label scarcity in scATAC-seq data. Additionally, our proposed cross-omic multi-sample Mixup mitigates cell heterogeneity in a sample-to-sample manner. By combining these aspects, DANCE significantly outperforms previous approaches by a large margin. Additionally, existing methods are data-hungry, heavily relying on cell type annotations. Therefore, when the number of labels decreases, the performance drops rapidly.In contrast, DANCE is data-efficient, capable of achieving excellent performance with fewer labels. Consequently, the performance decline due to label reduction is less pronounced, thanks to our proposed dual label scarcity elimination strategy. Due to the potential class imbalance issue in single-cell data, simply increasing the number of labels may not improve performance for some methods. For example, on the snRNA_SMARTer-snmC dataset, scBridge [36] achieves a performance of 36.15% under high label ratio, which is not better than that of 39.75% under mid label ratio. In contrast, our OT-based pseudo-labeling approach considers distributing pseudo-labels in a one-to-many manner, effectively preventing biased predictions. Therefore, on all datasets, increasing the number of labels leads to corresponding performance improvements for our DANCE. From the results, it can be observed that the performance of DANCE w/o OP significantly decreases. This indicates that OT-based pseudo-labeling effectively alleviates the label scarcity issue in scRNA-seq data by generating pseudo-labels for source scRNA-seq data and augmenting the scRNA-seq dataset. Additionally, the slight decrease in performance of DANCE w/o SL suggests that for heterogeneous scATAC-seq data, applying consistency regularization to source-like samples can further enhance model performance. Similarly, the minor decrease in performance of DANCE w/o TS suggests that target-specific samples should not be simply discarded as in previous methods. For these samples, collecting potentially correct cell types and removing them can provide weak supervision, thereby aiding performance improvement. Furthermore, the significant decrease in performance of DANCE w/o CM underscores the severity of the cell heterogeneity issue. Our proposed cross-omic multi-sample Mixup effectively eliminates the heterogeneous gap by generating mixed source scRNA-seq samples that simulate the distribution of target scATAC-seq data and aligning the two modalities. Thus, the effectiveness of all proposed modules is sufficiently validated.Sensitivity Analysis. In Figure 3 (a), we analyze the sensitivity of two crucial hyper-parameters k and λ on scRNA_SMARTer-ATAC. Firstly, we fix the other parameters to investigate the sensitivity of the model to different values of k. Here, k denotes the size of the partial label set and is used to control the number of potential correct cell types. As k varies within the range {2, 3, 4, 5, 6, 7}, the model's performance initially increases before gradually stabilizing. Considering all three scenarios, the optimal performance is achieved at k = 4 or k = 5. Next, with other parameters fixed, we analyze the sensitivity of the coefficient λ of L T S in Eqn. 10. Since L T S provides additional weak supervision by removing incorrect cell types, it typically requires a small coefficient. We change the value of λ within the range {0.05, 0.1, 0.15, 0.2, 0.25, 0.3}. Similarly, the model's performance firstly increases and then decreases, with better performance observed at λ = 0.1.t-SNE Visualization. In Figure 3 (b), we make the t-SNE [57] visualization to assess the heterogeneous gap of representations learned by scNCL [72] and our DANCE on scRNA_10X_v2-ATAC. Note that the higher overlap degree between the red and blue parts reflects the lower cell heterogeneity between scRNA-seq and scATAC-seq data. It can be observed that there is almost no overlap between the representations of scNCL [72] , indicating that it does not address the issue of cell heterogeneity.In contrast, the representations of our DANCE exhibit a high degree of overlap, indicating that our approach effectively mitigates cell heterogeneity even in the presence of label scarcity.A-distance Comparison. In Figure 3 (c), we compare the A-distance on scRNA_10X_v2-ATAC with various label ratios. The A-distance is calculated by employing a model and testing its ability to distinguish between the two modalities. Therefore, a low A-distance indicates a small heterogeneous gap. The A-distance is defined as: dist A = 2(1 -2ϵ), where ϵ is the test error. When ϵ = 0, the upper bound of dist A is 2. This indicates that there is a significant heterogeneous gap between the two modalities, and the model can easily differentiate between them. From the results, it can be observed that the A-distance of FixMatch [52] and scNCL [72] consistently remains close to 2, indicating a significant heterogeneity between scRNA-seq data and scATAC-seq data. In contrast, DANCE successfully reduces cell heterogeneity, resulting in a significant decrease in the A-distance. 
conclusion: 
 In this paper, we focus on a realistic yet underexplored problem of knowledge transfer across multiomic single-cell data under label scarcity in both scRNA-seq and scATAC-seq data, and propose a semi-supervised framework termed DANCE for this problem. Specifically, DANCE operates dual label scarcity elimination by OT-based dataset expansion for scRNA-seq data and a divide-and-conquer strategy for scATAC-seq data. DANCE further introduces a sample-to-sample alignment strategy of cross-omic multi-sample Mixup to reduce cell heterogeneity. Extensive experiments on various datasets verify the superiority of DANCE in comparison to many state-of-the-art approaches.Broader Impacts and Limitations. This study addresses the challenge of knowledge transfer across multi-omic single-cell data, which has significant potential for advancing biological research by reducing annotation costs in both source and target biological data. However, it is important to acknowledge that this work represents an initial exploration of this field and may have certain limitations. For instance, our method may not perform optimally in complex real-world scenarios, including open-set knowledge transfer and knowledge transfer under label noise. Additionally, this study primarily focuses on static scenarios, while there are complex dynamic applications, such as single-cell RNA velocity inference, that require further investigation. In future research, our goal is to address these challenges and expand our approach to encompass more generalized scenarios. To the best of our knowledge, no potential negative impacts resulting from our work have been identified. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: All the creators of assets are properly credited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: Proof of Theorem 4.1.(1) Let P be the projection ofEQUATIONThen, by definition, we haveEQUATIONwhere ⟨•, •⟩ is the standard matrix inner product. From definition, we know that P is the solution ofEQUATION)Using Lagrange multiplier, we know thatEQUATIONwhere β = (β 1 , . . . , β C ) T andEQUATIONFurther, using standard OT theory, we know that as σ → ∞, Q converges to the solution of the problem in Eqn. 17 with maximal entropy, Q ∞ . Therefore, for large enough σ, we know thatEQUATIONSo, using Cauchy's inequality, we haveEQUATION)Combining Eqn. 18-23 and take Q = Q ∞ in Eqn. 18, we get ⟨P n , -log P ⟩ ≥ ⟨Q, -log P ⟩ + C P -ε.Therefore, we haveEQUATION(2) For simplicity, we denote L P := (L ij /P ij ) B×C . Then, we haveEQUATIONHence,EQUATION)Proof of Theorem 4.2. From the definition, we know thatEQUATION)Therefore, Ep hard = EZ 1 = p+β -2pβ, Var(p hard ) = Var(Z 1 )/n, Ep t soft = (1-t)+(2t-1)EZ 1 , and Var(p t soft ) = (2t -1) 2 Var(Z 1 )/n. Hence, we haveEQUATION)DefineEQUATION)Then, from definition, we know that f (1) = 0. Further, we knowEQUATIONSince 2(1 -2EZ 1 ) 2 + 8Var(Z1) n > 0, we know that there existsEQUATIONsuch that f ′ (t) < 0 when t ∈ (c 0 , 1). So, f (t) ≥ f (1) = 0 for t ∈ (t 0 , 1). Hence,EQUATION)The step-by-step training algorithm of our DANCE is summarized in Algorithm 1. The model can get extra supervision from target-specific scATAC-seq data by removing the incorrect types from the candidate label set, the algorithm is provided in Algorithm 2.The training datasets D (s),l , D (s),u and D (t) . Ensure: The network parameters in H(F (•)); 1: Warm up the network using D (s),l ; 2: repeat 3:Update the pseudo-labels using Eqn. 3 and Eqn. 4;Expanding D (s) using Eqn. 5;5:for t = 1, 2, • • • , T do 6:Sample a mini-batch from the training datasets;7:Generate the predictions for both scRNA-seq data and scATAC-seq data by propagating the network;8:Calculate the final loss using Eqn. 16;9:Update the parameters of the network using backpropagation;10:end for 11: until convergenceAll the baselines are re-implemented on NVIDIA Tesla A100 40G GPUs using PyTorch according to the original settings in the corresponding papers to ensure a fair comparison. A two-layer MLP with an embedding dimension of 64 is employed as the encoder, and another two-layer MLP is adopted as the classifier. For all the baselines and our DANCE, we first warm up the model with labeled scRNA-seq data for 30 epochs and then train the model for another 30 epochs with a batch size of 32. Three settings with different label ratios (low: 1%, mid: 5%, high: 10%) are set up for experiments on each dataset to validate the sensitivity of these methods to the number of labels. We opt for SGD as the default optimizer with a learning rate of 3e -3 and a weight decay of 1e -3. for i = 1, 2, • • • , B T C do for y j i ∈ Y i do if Eqn. 8We summarize the dataset information included in experiments in Table 3 : Here we provide a brief introduction of the compared baseline methods as follows:• DAN [42] is a deep neural network (DNN) based method for domain adaptation, which adapts the layers and the corresponding task-specific features in a layer-wise manner.• CDAN [43] incorporates the discriminative information of the predictions from the classifier with adversarial learning for domain adaptation.• MCC [26] is a strong domain adaptation method that achieves fast convergence speed without the explicit use of domain alignment.• FixMatch [52] is a strong semi-supervised learning approach that hybrids pseudo-labeling and consistency regularization for self-training.• scJoint [38] is a multi-omic single-cell data integration method based on transfer learning and semi-supervised learning, which effectively transfers labels through kNN on joint embedding space.• scNCL [72] is a transfer learning framework that preserves the intrinsic structure of scATAC-seq data and employs contrastive learning to facilitate the transfer of cell types.• scBridge [36] integrates multi-omic single-cell data in a heterogeneous manner by mining reliable cross-modal samples and expanding the single-cell dataset. As depicted in Figure 4 , classic pseudo-labeling [52] assigns the category label with the highest confidence as the pseudo-label, establishing a one-to-one correspondence between a sample and a category label. This approach overlooks the potential connections between the sample distribution and the class distribution, especially in cases of class imbalance. In contrast, our proposed OT-based pseudo-labeling aligns multiple samples with a category center through the coupling matrix Q and the cost matrix -logP . This approach considers the connections between the sample and class distribution, effectively addressing the issue of potential biased predictions in scRNA-seq data. Different from traditional approaches [52, 78] that directly discard the target-specific samples, we utilize these samples to provide weak supervision for the model (algorithm can be found in Algorithm 2). By progressively removing the incorrect cell types, the model gets extra guidance from these samples. Incorporating consistency regularization for source-like samples (Figure 5 ) and incorrect cell types removal for targetspecific samples, the entire scATACseq dataset D (t) = D SL ∪ D T C is fully explored, contributing to the semi-supervised knowledge transfer process. Different from previous alignment strategies that rely on labels from scATAC-seq data or paired labels, our proposed cross-omic multi-sample Mixup reconstructs class-wise similarity through the weights of classifiers (Figure 6 ). It then fuses scRNA-seq samples by incorporating semantic information from scATACseq data. Without the need for labels from scATAC-seq data or pairwise labels, we transform the distribution of scRNA-seq samples to align with scATAC-seq data. This makes our alignment strategy more flexible and convenient. Considering the high cost of annotating singlecell data, our alignment strategy is more suitable for the problem of semi-supervised knowledge transfer across multi-omic single-cell data. In addition to these parameters, we also investigate the two empirical hyper-parameters: σ and τ .From the results in Table 4 and Table 5 , we can observe that the performance is not sensitive to the choice of σ when we change σ from 5 to 20, so we empirically set its value to 10. We also vary the value of τ with the range of {0.8, 0.85, 0.9, 0.95}. The results below indicate that the method is not sensitive to τ in the interval [0.8, 0.95]. Therefore, we set τ to 0.9 as the default. We progressively remove the incorrect cell types for additional supervision. However, how to define an incorrect cell type may be a point of interest. To answer this question, We have included a sensitivity analysis of the threshold µ by varying it in {0.0005, 0.001, 0.0015, 0.002}. The results in Table 6 indicate that a threshold of 1e -3 brings good performance with significant differences. In particular, after the softmax operation, the sum of the confidences for all cell types equals 1 with Besides scRNA_10X_v2-ATAC, we further compare the A-distance on 3 more datasets, including scRNA_10X_v2-snmC, snRNA_10X_v2-ATAC, and snRNA_10X_v2-snmC in Figure 10 . The results are consistent with the analysis in Section 5.2, which validates the robustness of our approach in reducing cell heterogeneity.We showcase additional qualitative experimental results on two more datasets in Figure 7 . It can still be observed that our DANCE achieves the best performance. From the results, we can obtain consistent conclusions as previously mentioned in Section 5.2.In Figure 8 and Figure 9 , we make the t-SNE visualization comparison with more baseline methods.From the results in the first row, it can be found that there is almost no overlap degree in FixMatch [52] and scNCL [72] . There are slight overlaps in scBridge [36] , but the heterogeneous gap remains large when certain parts of labels are missing. Compared with these methods, our DANCE achieves the largest overlap degree since it effectively reduces cell heterogeneity. In the second row, each different color represents a different cell type. It can be seen that, in the case of label scarcity, our DANCE still Despite the effectiveness of OT-based dataset expansion for source scRNA-seq data, due to the presence of cellular heterogeneity, directly applying this strategy to target single-cell data may not be effective. We have added a model variant w/ OT for scATAC-seq to support our point. The compared results on CITE-ASAP dataset are shown in Table 7 . We can find that our full model outperforms w/ OT for scATAC-seq, which validates the OT strategy may not be the optimal choice given the significant cellular heterogeneity inherent in scATAC-seq data. This is why we introduce the divide-and-conquer strategy for target samples. An interesting question is whether the initial prediction's reliance on the divide-and-conquer strategy has been accurately labeled. To explore this issue, we have included a comparison of pseudo-labeling accuracy between unlabeled scRNA-seq data and source-like scATAC-seq data in Table 8 . The results for the snRNA_SMARTer-snmC dataset are presented below. From the findings, it is evident that the accuracy of the source-like scATAC-seq data closely aligns with that of unlabeled scRNA-seq data, confirming the effectiveness of our approach. Even though mainstream algorithms focus on transferring cell type knowledge from scRNA-seq to scATAC-seq data, we are interested in exploring the possibility of cell type transfer in the opposite direction, namely from scATAC-seq to scRNA-seq. We have conducted an experiment that performs this reverse cell type transfer on the CITE-ASAP dataset. The results presented in Table 9 confirm the effectiveness of our method in this reverse direction. The comparison of computation costs is also a point of interest. We have included a comparison of memory and time in Table 10 , and it is evident that our method offers a competitive computation cost. Specifically, the performance of scNCL is significantly inferior to ours (our performance improvement exceeds 105.9%), while our memory cost sees only a slight increase with even less training time. To benchmark our DANCE against more baseline methods, we have included SLA [76] , COT [41] , Seurat [54] , and Harmony [30] on different datasets for performance comparison. The results shown in Table 11 demonstrate the effectiveness of the proposed DANCE. In real-world scenarios, the proposed DANCE can transfer cell types from multiple-source scRNA-seq data to target scATAC-seq data. We have included experiments to transfer cell type knowledge from a labeled scRNA-seq dataset and an unlabeled scRNA-seq dataset to scATAC-seq data. The results in Table 13 indicate that our approach is superior to other methods in practical scenarios.• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 10. Broader Impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [Yes] Justification: The discussion about the broader impacts is provided in the conclusion section and appendix. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).We use scATAC-seq as an example and there could be other biological layers.https://tabula-muris.ds.czbiohub.org/ 


Decision: This paper proposes a method to transfer knowledge from single-cell RNA sequencing (scRNA-seq) to single-cell ATAC-sequencing (scATAC-seq), a problem that can be formulated as domain adaptation. The proposed method, Dual Label Scarcity Elimination with Cross-omic Multi-sample Mixup, or DANCE for short, achieves this knowledge transfer through several steps, including expanding the labeled RNA data via pseudo-labeling and using a confidence-based divide-and-conquer algorithm to identify easy/hard scATAC-seq samples with respect to scRNA-seq, thereby efficiently transferring knowledge to the ATAC sequencing domain.

The paper initially received scores of 2x6 (weak accept), 1x5 (borderline accept), and 1x4 (borderline reject). After the rebuttal and during the author-reviewer discussion period, the scores improved to 2x7 (accept), 1x5 (borderline accept), and 1x4 (borderline reject). The reviewers acknowledged the novelty of the idea within the context of the problem, and the AC agrees with this assessment, recommending the paper for acceptance. **Congratulations**.

The AC suggests including all the results discussed during the rebuttal and the author-reviewer discussions. Additionally, the following two aspects should be addressed:

1. Provide an intuitive explanation of the theorems and their implications in the main text. Currently, the theorems appear somewhat disconnected, and an intuitive explanation will help readers appreciate them better.

2. The choice of marginals in the optimal transport (OT) implies that mini-batches should be uniform across the class distribution. This should be discussed and explained.

Congratulations again.

Reviews: review: paper_summary: paper_summary: This paper addresses the challenge of knowledge transfer across multi-omic single-cell data under label scarcity. The proposed semi-supervised framework, DANCE, uses optimal transport to generate pseudo-labels and a divide-and-conquer strategy for handling scATAC-seq data. The framework demonstrates superior performance on benchmark datasets, offering a practical solution to label scarcity in multi-omic data.
strength_weakness: strength_weakness: **Cons:**
1. The exclusion of OT in scATAC-seq data due to cellular heterogeneity needs practical examples or quantitative analysis to substantiate its significance.
2. The initial prediction's dependency in the divide-and-conquer strategy could lead to misclassification. Discussion on handling wrongly divided samples or providing statistics is needed.
3. Exploring if DANCE can be conducted in the opposite direction (scATAC-seq with OT and scRNA-seq with divide-and-conquer) would be beneficial.
4. Discussion on the complexity of OT and divide-and-conquer in terms of memory and time compared to existing studies is required.
5. Including a discussion on the use of scCLIP [1] in scenarios with scarce labels would enhance the paper.

[1] https://openreview.net/forum?id=KMtM5ZHxct
questions: questions: See the Weaknesses above.
limitations: limitations: I do not observer any potential negative societal impact of this work.
paper_summary: paper_summary: This paper introduces a semi-supervised knowledge transfer framework called DANCE, designed to effectively transfer cell type annotations from scRNA-seq data to unannotated scATAC-seq data under conditions of label scarcity. It is similar to the unsupervised domain adaptation task in computer vision. DANCE addresses the challenge of heterogeneous multi-omic data by generating pseudo-labels based on optimal transport, employing a divide-and-conquer strategy, and using cross-omic multi-sample Mixup to reduce cell heterogeneity. Extensive experiments demonstrate DANCE's superiority over state-of-the-art methods.
strength_weakness: strength_weakness: 1. The compared methods, especially the DA method, are relatively old. Comparison with newer methods (e.g., [a], [b], etc.) helps to understand the performance of the methods.
2. Repeated use of symbols may cause confusion and misunderstanding among readers. Eq. (1), (3), (16), and Theory 3.2 all contain $\lambda$. As far as I understand, the meanings of these symbols may be different.
3. The method includes too many empirical hyperparameters, and the ‘crucial’ parameters selected by parameter analysis seem not comprehensive enough. A more comprehensive description of each $\lambda$ and threshold $\tao$ will help to understand the method and promote future work.
4. The quality of images seems to be low, affecting comprehension. To be specific, the method process is not clearly and comprehensively shown in Figure 1. The text in Figure 2 is too small.

[a] Semi-Supervised Domain Adaptation with Source Label Adaptation
[b] COT: Unsupervised Domain Adaptation with Clustering and Optimal Transport
questions: questions: Please refer to the strengths and weaknesses of the paper. In addition, in my opinion, the proposed method seems to be a patchwork of methods in domain adaptation, cross-modal alignment, etc. Maybe its technical innovation is questionable. A more targeted and detailed description is recommended.
limitations: limitations: The author has explained the limitations in Sec. 5.
paper_summary: paper_summary: This paper proposes a label transfer method from scRNA-seq to scATAC-seq data. Based on the heterogeneity of single-cell data, this work partitions data into several groups and designs effective strategies to tackle them respectively. Experiments demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: 1. In the Introduction, the authors claim that "a small fraction of scRNA-seq data with cell types annotated" aligns more closely with practical scenarios. Do you have any evidence to support the claim? You may provide an example to show that scRNA-seq data are not processed and annotated as a whole, but only a small portion is annotated.
2. The uniform distribution assumption in the optimal transport process could be wrong, as most scRNA-seq data are unbalanced across different cell types.
3. In Eq. 8, the negative cell types are removed with a significant difference from the cell type with the highest probability. However, does a threshold of 1e-3 represent significant differences?
4. What does $S_i$ in Eq. 10 mean?
5. What is the motivation of mixing scRNA-seq data to form scATAC-seq data? Is such an operation biologically reasonable?
6. Typo: In line 253 cBridge -> scBridge
7. Currently the author split the MouseAtlas dataset into several subsets with different combinations for evaluation. I recommend the author provide the experimental results on the full dataset.
questions: questions: Please refer to my concerns in the weaknesses section.
limitations: limitations: No significant limitations of this work are found.
paper_summary: paper_summary: This paper introduces a novel method to address the knowledge transfer challenge in multi-omic single-cell data. Specifically, it focuses on scenarios where annotations are available partially in one modality, namely scRNA-seq data, and aims to infer annotations in another modality, the scATAC-seq data, without requiring paired datasets.

The authors proposed to use a shared encoder to project the two modalities, with additional compoenents including: optimal transport-based dataset expansion for scRNA-seq data, divide and conpuer for target scATAV-seq data, and cross-omic multi-sample mixup. 

The author conducted comprehensive comparison with SOTA method and showed that the proposed method achieves the best performance.
strength_weakness: strength_weakness: I'm not entirely convinced about the prevalence of the problem setting described by the author, where only a small subset of the scRNA-seq data is annotated. It may occur in scenarios such as large experiments conducted in batches, where only certain batches are annotated. It raises questions about the experimental setup. Are annotations removed randomly or based on specific conditions or batches? I wonder how does the availability of annotated subsets (randomly or by batch) affect the overall study results.
questions: questions: I wonder how the methodology comparision is conducted. As the authors argued in the paper, the SOTA methods in the experimental studies does not deal with scenarios where the scRNA-seq is only partially annotated. Therefore, in the comparison experiment, such as in the "low label ratio" setup, is it that only the 1% of the scRNA-seq is used for cross-modality matching? There are popular existing methods such as Seurat and Harmony that provided a potential two-step approach: first learn the full annotation on scRNA-seq, and then transfer to scATAC-seq, which might be a more fair comparison.
limitations: limitations: The authors have addressed limitations such as only applicable to open-set settings.

Comments: title: Summary of Rebuttal
Thanks for your feedback and increasing your score!
Thank you for your feedback!
Thanks for your feedback and increasing the rating!
Thank you for your feedback!
Thank you for your feedback!
Thank you for your feedback!
comment: Dear Reviewers and Area Chairs,

We would like to express our sincere gratitude for your great efforts, insightful comments, support, and the constructive suggestions you have provided once again! Through our discussions and the reviewers' responses, it appears that we have effectively addressed the major concerns raised by reviewers, and received higher scores from Reviewer W1Pi and Reviewer puzU. This outcome has greatly benefited us, and we would like to thank all of you for your valuable support!

The reviewers held many positive comments on our paper, including "**solid theory**” (Reviewer kGak), "effective and well-handled method" (Reviewer W1Pi), "**impressive performance**" (Reviewer W1Pi), “well-written and organized” (Reviewer W1Pi, Reviewer puzU), "well-motivated and reproducible" (Reviewer kGak), "well-structured and easy to follow” (Reviewer kGak), "good experiment” (Reviewer kGak), "technically sound method” (Reviewer puzU), “**effective ablation studies**” (Reviewer puzU, Reviewer L3nP), and “**innovative approach**” (Reviewer L3nP).

The reviewers also raised insightful and constructive concerns. We made every effort to address all the concerns by providing detailed clarification and requested results. Here is the summary of the major revisions:

* We have added more **ablation studies** and **parameter analysis** to show the effectiveness and reliability of each component including OT and divide-and-conquer strategy.
* We have added **efficiency analysis** to make the paper more complete.
* We have included **more competing baselines** including SLA, COT, Seurat, and Harmoy to demonstrate the superiority of our approach.
* We have provided **more references** and **real-world examples** to clarify the practical usage of our method and the rationale of our settings.
* We have demonstrated **more performance comparisons** in different experimental settings to validate the effectiveness of the proposed method in the real world.

We firmly believe that our framework for knowledge transfer across multi-omic single-cell data plays a significant role in advancing the community. And we are committed to making our complete code and training details publicly available. All the rebuttal contents will be properly included in the final version, following your valuable suggestions. Moreover, we are eager to engage in further discussions with you to enhance our understanding of the domain and further improve the quality of the paper.

Once again, thank you for your time and effort in reviewing our work. We greatly appreciate your assistance in improving our manuscript!

Best regards,

the Authors
Thanks for your feedback and increasing your score! We are pleased to know that your concerns have been alleviated. We will properly include all the rebuttal contents in the revised version, following your valuable suggestions.
Thank you for all the efforts on rebuttal. My concerns have been alleviated, and I would like to increase my score to 7.
Thanks for your feedback! We are pleased to know that you keep the positive rating and your support. We will properly include all the rebuttal contents in the revised version, following your valuable suggestions.
I appreciate the effort put into the rebuttal, which addressed some of my concerns. After reading the other reviews and replies, I  keep my positive rating.
Thank you for your feedback and increasing the rating! We are pleased to know that our responses have addressed your concerns. We will properly include all the rebuttal contents in the final version, following your valuable suggestions.
Thanks for the response. Now I am more convinced about the effectiveness of the proposed method, and touched by your effort in rebuttal. I will raise my score to accept.
Thank you for your feedback! We are pleased to address your further questions as follows:


>Q1. Thanks for the response. In that case, a more practical simulation would be, for example, labeling the scRNA-seq data from one source while leaving scRNA-seq data from other sources to be unlabeled.

A1. Thanks for your suggestion. We have included experiments to transfer cell type knowledge from a labeled scRNA-seq dataset and an unlabeled scRNA-seq dataset to scATAC-seq data. The following results indicate that our approach is superior to other methods in practical scenarios.

| Labeled scRNA-seq Data   | CITE           | CITE          | snRNA_10X_v3_A | snRNA_SMARTer |
| ------------------------ | -------------- | ------------- | -------------- | ------------- |
| Unlabeled scRNA-seq Data | snRNA_10X_v3_A | snRNA_SMARTer | CITE           | CITE          |
| scATAC-seq Data          | ASAP           | ASAP          | ASAP           | ASAP          |
| scJoint                  | 86.59          | 87.27         | 75.94          | 76.53         |
| scBridge                 | 88.01          | 89.24         | 78.99          | 77.60          |
| scNCL                    | 86.24          | 88.28         | 75.80          | 75.42         |
| Ours                     | **93.37**      | **92.48**     | **84.88**      | **83.79**     |



Thank you again for your feedback and effort! Please let us know if you have further questions.
Thanks for the response. In that case, a more practical simulation would be, for example, labeling the scRNA-seq data from one source while leaving scRNA-seq data from other sources to be unlabeled.
Thank you for your feedback! We are pleased to address your further questions as follows:


>Q1. Thanks for the further clarification. I totally understand that annotating single-cell data could be laboring. However, in the current setting, you randomly sample a subset to have labels. My concern is that, in practice, one dataset is more likely to be annotated as a whole, instead of sampling a 1% subset for annotation. Could you provide more explanation and evidence on that?

A1. Thanks for your question. 

*Firstly*, we could have a completely new dataset without any annotation in the real world. In such a scenario, we need to make an effort to label the new data by ourselves and the proposed method can serve as an economic tool to save the labeling cost.

*Secondly*, to ensure a fair and effective evaluation of different algorithms, we rely on fully labeled datasets and randomly sample a subset. Otherwise, we do not know the ground truth on test data for performance comparison. This evaluation strategy can be found in previous semi-supervised single-cell analysis works [1,2,3,4]. 

**Reference**


[1] Dong et al., scSemiAE: a deep model with semi‑supervised learning for single‑cell transcriptomics, BMC Bioinformatics, 2022  
[2] Dong et al., Semi-supervised Deep Learning for Cell Type Identification from Single-Cell Transcriptomic Data, IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2022  
[3] Chen et al., Single-cell RNA-seq data semi-supervised clustering and annotation via structural regularized domain adaptation, Bioinformatics, 2021  
[4] We et al., CALLR: a semi-supervised cell-type annotation method for single-cell RNA sequencing data, Bioinformatics, 2021  

Thank you again for your feedback and effort! Please let us know if you have further questions.
Thanks for the further clarification. I totally understand that annotating single-cell data could be laboring. However, in the current setting, you randomly sample a subset to have labels. My concern is that, in practice, one dataset is more likely to be annotated as a whole, instead of sampling a 1% subset for annotation. Could you provide more explanation and evidence on that?
Thank you for your feedback! We are pleased to address your further questions as follows:

>Q1. Thanks for the detailed responses which addressed most of my concerns. However, the current evidence of whether the setting aligns with practical scenarios is still weak. It would be more convincing if the authors could provide some real-world examples. 

A1. Thanks for your suggestion. Here we provide some real-world examples. 

The annotation of single-cell data is quite challenging and time-consuming. For example, in PanglaoDB [1], the assignment of genes to cell types requires broad expertise to make mappings from gene markers to cell types. The complexity is further amplified by the exhaustive list of cell type markers spanning 11 columns, making the annotation process intricate and less efficient. Moreover, ground truth cell-type annotation often requires experimental validation techniques such as flow cytometry [2] and copy number variations (CNV) estimation method [3,4]. Therefore, our semi-supervised methods can save the experimental and labor costs when it comes to new datasets in the real world. 

Based on these examples, we can conclude that label scarcity in scRNA-seq data is a practical problem in real-world scenarios. We will include it in the revised version.

>Q2. Moreover, the experimental results on full datasets are more solid than those on subsets. They should be included in the main paper after the revision.

A2. Thanks for your suggestion. We will definitely include the experimental results in the revised version.

**Reference**

[1] Franzén et al., PanglaoDB: a web server for exploration of mouse and human single-cell RNA sequencing data, Database, 2019  
[2] Zhang et al., Regulatory T-cell depletion alters the tumor microenvironment and accelerates pancreatic carcinogenesis, Cancer Discov, 2020  
[3] Zhang et al., Single-cell analyses inform mechanisms of myeloid-targeted therapies in colon cancer, Cell, 2020  
[4] Kim et al., Single-cell rna sequencing demonstrates the molecular and cellular reprogramming of metastatic lung adenocarcinoma, Nature Communication, 2020


Thank you again for your feedback and effort! We will add the rebuttal contents to the main paper in the final version following your valuable suggestions. Please let us know if you have further questions.
Thanks for the detailed responses which addressed most of my concerns. However, the current evidence of whether the setting aligns with practical scenarios is still weak. It would be more convincing if the authors could provide some real-world examples. Moreover, the experimental results on full datasets are more solid than those on subsets. They should be included in the main paper after the revision.