Original: Graph Neural Networks for Multimodal Single-Cell Data Integration
Revised: Graph Neural Networks for Multimodal Single-Cell Data Integration

Original: Abstract
Revised: Abstract

Original: Recent advances in multimodal single-cell technologies have enabled simultaneous acquisitions of multiple omics data from the same cell, providing deeper insights into cellular states and dynamics.
Revised: Recent advances in multimodal single-cell technologies have enabled simultaneous acquisitions of multiple omics data from the same cell, providing deeper insights into cellular states and dynamics.

Original: However, it is challenging to learn the joint representations from the multimodal data, model the relationship between modalities, and, more importantly, incorporate the vast amount of single-modality datasets into the downstream analyses.
Revised: However, it is challenging to learn the joint representations from the multimodal data, model the relationship between modalities, and, more importantly, incorporate the vast amount of single-modality datasets into the downstream analyses.

Original: To address these challenges and correspondingly facilitate multimodal single-cell data analyses, three key tasks have been introduced:
Revised: To address these challenges and correspondingly facilitate multimodal single-cell data analyses, three key tasks have been introduced:

Original: Modality prediction , Modality matching and Joint embedding .
Revised: Modality prediction , Modality matching and Joint embedding .

Original: In this work, we present a general Graph Neural Network framework scMoGNN to tackle these three tasks and show that scMoGNN demonstrates superior results in all three tasks compared with the state-of-the-art and conventional approaches.
Revised: In this work, we present a general Graph Neural Network framework scMoGNN to tackle these three tasks and show that scMoGNN demonstrates superior results in all three tasks compared with the state-of-the-art and conventional approaches.

Original: Our method is an official winner in the overall ranking of Modality prediction from NeurIPS
Revised: Our method is an official winner in the overall ranking of Modality prediction from NeurIPS

Original: Competition 2 , and all implementations of our methods have been integrated into
Revised: Competition 2 , and all implementations of our methods have been integrated into

Original: DANCE package[8] 3 .
Revised: DANCE package 3 .

Original:  Introduction
Revised:  Introduction

Original: The rapid advance of single-cell technologies makes it possible to simultaneously measure multiple molecular features at multiple modalities in a cell, such as gene expressions, protein abundance and chromatin accessibility.
Revised: The rapid advance of single-cell technologies makes it possible to simultaneously measure multiple molecular features at multiple modalities in a cell, such as gene expressions, protein abundance and chromatin accessibility.

Original: For instance, CITE-seq (cellular indexing of transcriptomes and epitopes by sequencing) [29] enables simultaneous quantification of mRNA expression and surface proteins abundance; methods like sci-CAR [4], Paired-seq [36] and SNARE-seq [6] enable joint profiling of mRNA expression and chromatin accessibility ( i e. genome-wide DNA accessibility).
Revised: For instance, CITE-seq (cellular indexing of transcriptomes and epitopes by sequencing) [29] enables simultaneous quantification of mRNA expression and surface proteins abundance; methods like sci-CAR [4], Paired-seq [36] and SNARE-seq [6] enable joint profiling of mRNA expression and chromatin accessibility ( i e. genome-wide DNA accessibility).

Original: The joint measurements from these methods provide unprecedented multimodal data for single cells, which has given rise to valuable insights for not only the relationship between different modalities but, more importantly, a holistic understanding of the cellular system.
Revised: The joint measurements from these methods provide unprecedented multimodal data for single cells, which has given rise to valuable insights for not only the relationship between different modalities but, more importantly, a holistic understanding of the cellular system.

Original: Despite the emergence of joint platforms, single-modality datasets are still far more prevalent.
Revised: Despite the emergence of joint platforms, single-modality datasets are still far more prevalent.

Original: How to effectively utilize complementary information from multimodal data to investigate cellular states and dynamics and to incorporate the vast amount of single-modality data while leveraging the multimodal data pose great challenges in single-cell genomics.
Revised: How to effectively utilize complementary information from multimodal data to investigate cellular states and dynamics and to incorporate the vast amount of single-modality data while leveraging the multimodal data pose great challenges in single-cell genomics.

Original: To address these challenges, Luecken et al.
Revised: To address these challenges, Luecken et al.

Original: [10] summarized three major tasks: (1) Modality prediction aims at predicting the features of one modality from the features of another modality [33]; (2) Modality matching focuses on identifying the correspondence of cells between different modalities [32]; and (3) Joint embedding requires embedding the features of two modalities into the same low-dimensional space [29].
Revised: [10] summarized three major tasks: (1) Modality prediction aims at predicting the features of one modality from the features of another modality [33]; (2) Modality matching focuses on identifying the correspondence of cells between different modalities [32]; and (3) Joint embedding requires embedding the features of two modalities into the same low-dimensional space [29].

Original: The motivation of modality prediction and modality matching is to better integrate existing single-modality datasets, while joint embedding can provide more meaningful representations of cellular states from different types of measurements.
Revised: The motivation of modality prediction and modality matching is to better integrate existing single-modality datasets, while joint embedding can provide more meaningful representations of cellular states from different types of measurements.

Original: In light of these benefits, computational biologists recently organized a competition for multimodal single-cell data integration at NeurIPS 2021 [10] to benchmark these three tasks and facilitate the computational biology communities.
Revised: In light of these benefits, computational biologists recently organized a competition for multimodal single-cell data integration at NeurIPS 2021 [10] to benchmark these three tasks and facilitate the computational biology communities.

Original: There is an emerging trend to leverage deep learning techniques to tackle the tasks mentioned above for multimodal single-cell data [21].
Revised: There is an emerging trend to leverage deep learning techniques to tackle the tasks mentioned above for multimodal single-cell data [21].

Original: BABEL [33] translated between the transcriptome (mRNA) and chromatin (DNA) profiles of a single cell based on an encoder-decoder architecture; scMM[20] implemented a mixture-of-experts deep generative model for joint embedding learning and modality prediction.
Revised: BABEL [33] translated between the transcriptome (mRNA) and chromatin (DNA) profiles of a single cell based on an encoder-decoder architecture; scMM[20] implemented a mixture-of-experts deep generative model for joint embedding learning and modality prediction.

Original: Cobolt[12] acquired joint embedding via a variant of Multimodal Variational
Revised: Cobolt[12] acquired joint embedding via a variant of Multimodal Variational

Original: Autoencoder (MVAE[35]).
Revised: Autoencoder (MVAE[35]).

Original: MOFA2[1] used Bayesian group factor analysis to reduce dimensions of multi-modality data and generate a low-dimensional joint representation.
Revised: MOFA2[1] used Bayesian group factor analysis to reduce dimensions of multi-modality data and generate a low-dimensional joint representation.

Original: However, most of these approaches treat each cell as a separate input without considering possible high-order interactions among cells or different modalities.
Revised: However, most of these approaches treat each cell as a separate input without considering possible high-order interactions among cells or different modalities.

Original: Such higher-order information can be essential for learning with high-dimensional and sparse cell features, which are common in single-cell data.
Revised: Such higher-order information can be essential for learning with high-dimensional and sparse cell features, which are common in single-cell data.

Original: Take the joint embedding task for example, the feature dimensions for GEX (mRNA) and ATAC (DNA) data are as high as 13,431 and 116,490, respectively; however, only 9 .
Revised: Take the joint embedding task for example, the feature dimensions for GEX (mRNA) and ATAC (DNA) data are as high as 13,431 and 116,490, respectively; however, only 9 .

Original: 75% of GEX and 2 .
Revised: 75% of GEX and 2 .

Original: 9% of ATAC features are nonzero on average over 42 , 492 training samples (cells).
Revised: 9% of ATAC features are nonzero on average over 42 , 492 training samples (cells).

Original: Furthermore, integrated measuring often requires additional processing to cells, which can lead to extra noise and drop-out in the resulting data [15, 19].
Revised: Furthermore, integrated measuring often requires additional processing to cells, which can lead to extra noise and drop-out in the resulting data [15, 19].

Original: Therefore, it is a desired technique that can mitigate the negative impact of such noise.
Revised: Therefore, it is a desired technique that can mitigate the negative impact of such noise.

Original: Recently, the advances in graph neural networks (GNNs) [14, 34, 3, 11, 17] pave the way for addressing the aforementioned issues in single-cell data integration.
Revised: Recently, the advances in graph neural networks (GNNs) [14, 34, 3, 11, 17] pave the way for addressing the aforementioned issues in single-cell data integration.

Original: Specifically, GNNs aggregate information from neighborhoods to update node embeddings iteratively [11].
Revised: Specifically, GNNs aggregate information from neighborhoods to update node embeddings iteratively [11].

Original: Thus, the node embedding can eventually encode high-order structural information through multiple aggregation layers.
Revised: Thus, the node embedding can eventually encode high-order structural information through multiple aggregation layers.

Original: In addition, GNNs smooth the features by aggregating neighbors’ embedding and also filter the eigen-values of graph Laplacian, which provides an extra denoising mechanism [18].
Revised: In addition, GNNs smooth the features by aggregating neighbors’ embedding and also filter the eigen-values of graph Laplacian, which provides an extra denoising mechanism [18].

Original: Hence, by modeling the interactions between cells and their features as a graph, we can adopt GNNs to exploit the structural information and tackle the limitations of previous techniques for single-cell data integration.
Revised: Hence, by modeling the interactions between cells and their features as a graph, we can adopt GNNs to exploit the structural information and tackle the limitations of previous techniques for single-cell data integration.

Original: With the constructed graph, we can readily incorporate external knowledge (e.g., interactions between genes) into the graph to serve as additional structural information.
Revised: With the constructed graph, we can readily incorporate external knowledge (e.g., interactions between genes) into the graph to serve as additional structural information.

Original: Given those advantages, we aim to design a GNN framework for multimodal data integration.
Revised: Given those advantages, we aim to design a GNN framework for multimodal data integration.

Original: While several existing works attempted to introduce graph neural networks to single cell analysis [27, 31, 7, 24], none of them tackle the challenging problem of multimodal data integration which requires handling different modalities simultaneously.
Revised: While several existing works attempted to introduce graph neural networks to single cell analysis [27, 31, 7, 24], none of them tackle the challenging problem of multimodal data integration which requires handling different modalities simultaneously.

Original: Therefore, we aim to develop GNN methods for the tasks in multimodal data integration, especially for modality prediction, modality matching and joint embedding.
Revised: Therefore, we aim to develop GNN methods for the tasks in multimodal data integration, especially for modality prediction, modality matching and joint embedding.

Original: Specifically, we propose a general framework scMoGNN for modeling interactions of modalities and leveraging GNNs in single-cell analysis 4 .
Revised: Specifically, we propose a general framework scMoGNN for modeling interactions of modalities and leveraging GNNs in single-cell analysis 4 .

Original: Our framework is highly versatile: we demonstrate its use cases in the three different multimodal tasks.
Revised: Our framework is highly versatile: we demonstrate its use cases in the three different multimodal tasks.

Original: To the best of our knowledge, we are the first to develop a GNN framework in multimodal single-cell data integration.
Revised: To the best of our knowledge, we are the first to develop a GNN framework in multimodal single-cell data integration.

Original: Our proposed framework achieves the best results in all of these three tasks on the benchmark datasets, providing a very strong baseline for follow-up research.
Revised: Our proposed framework achieves the best results in all of these three tasks on the benchmark datasets, providing a very strong baseline for follow-up research.

Original: Our contributions can be summarized as follows:
Revised: Our contributions can be summarized as follows:

Original: We study multimodal single-cell data integration and propose a general GNN-based framework scMoGNN to capture the high-order structural information between cells and modalities.
Revised: We study multimodal single-cell data integration and propose a general GNN-based framework scMoGNN to capture the high-order structural information between cells and modalities.

Original: Our scMoGNN is highly flexible as it can be adopted in different multimodal single-cell tasks.
Revised: Our scMoGNN is highly flexible as it can be adopted in different multimodal single-cell tasks.

Original: Our framework achieves remarkable performance across tasks.
Revised: Our framework achieves remarkable performance across tasks.

Original: It has won the first place of the modality prediction task in a 2021 NeurIPS competition, and currently outperforms other models for all three tasks.
Revised: It has won the first place of the modality prediction task in a 2021 NeurIPS competition, and currently outperforms other models for all three tasks.

Original:  Related Work
Revised:  Related Work

Original: GNNs on Single-Modality Data.
Revised: GNNs on Single-Modality Data.

Original: Graphs occur as a natural representation of single-cell data both as feature-centric (RNAs, DNAs, or proteins) and cell-centric.
Revised: Graphs occur as a natural representation of single-cell data both as feature-centric (RNAs, DNAs, or proteins) and cell-centric.

Original: Thus, a few recent works have applied
Revised: Thus, a few recent works have applied

Original: GNNs to the single-cell data.
Revised: GNNs to the single-cell data.

Original: Song et al.
Revised: Song et al.

Original: [27] propose scGCN model for knowledge transfer in single-cell omics (mRNA or DNA) based on Graph Convolutional Networks [14].
Revised: [27] propose scGCN model for knowledge transfer in single-cell omics (mRNA or DNA) based on Graph Convolutional Networks [14].

Original: scGNN [31] formulates and aggregates cell-cell relationships with Graph Neural Networks for missing-data imputation and cell clustering using single-cell RNA sequencing (scRNA-seq) data.
Revised: scGNN [31] formulates and aggregates cell-cell relationships with Graph Neural Networks for missing-data imputation and cell clustering using single-cell RNA sequencing (scRNA-seq) data.

Original: scDeepSort [24] is a pre-trained cell-type annotation tool for scRNA-seq data that utilizes a deep learning model with a weighted GNN.
Revised: scDeepSort [24] is a pre-trained cell-type annotation tool for scRNA-seq data that utilizes a deep learning model with a weighted GNN.

Original: Similar to our proposed model, scDeepSort also relies on feature-cell graphs.
Revised: Similar to our proposed model, scDeepSort also relies on feature-cell graphs.

Original: However, it does not incorporate any prior knowledge into GNNs.
Revised: However, it does not incorporate any prior knowledge into GNNs.

Original: Using spatial transcriptomics (mRNA) data, DSTG [26] utilizes semi-supervised GCN to deconvolute the relative abundance of different cell types at each spatial spot.
Revised: Using spatial transcriptomics (mRNA) data, DSTG [26] utilizes semi-supervised GCN to deconvolute the relative abundance of different cell types at each spatial spot.

Original: Despite its success on single-modality data, there are few efforts on applying GNNs to multimodal single-cell data.
Revised: Despite its success on single-modality data, there are few efforts on applying GNNs to multimodal single-cell data.

Original: Multimodal Data Integration.
Revised: Multimodal Data Integration.

Original: Most of the prior works in multimodal data integration can be divided into 1) matrix factorization [9, 28, 13] or statistical based methods [30, 25, 32] and 2) autoencoder based methods [33, 12].
Revised: Most of the prior works in multimodal data integration can be divided into 1) matrix factorization [9, 28, 13] or statistical based methods [30, 25, 32] and 2) autoencoder based methods [33, 12].

Original: Specifically, BABEl [33] leverages autoencoder frameworks with two encoders and two decoders to take only one of these modalities and infer the other by constructing reconstruction loss and cross-modality loss.
Revised: Specifically, BABEl [33] leverages autoencoder frameworks with two encoders and two decoders to take only one of these modalities and infer the other by constructing reconstruction loss and cross-modality loss.

Original: Cobolt[12] acquires joint embedding via a variant of Multimodal Variational Autoencoder (MVAE[35]).
Revised: Cobolt[12] acquires joint embedding via a variant of Multimodal Variational Autoencoder (MVAE[35]).

Original: Unlike our proposed models, these aforementioned methods are unable to incorporate high-order interactions among cells or different modalities.
Revised: Unlike our proposed models, these aforementioned methods are unable to incorporate high-order interactions among cells or different modalities.

Original: To the best of our knowledge, we are the first to apply GNNs in the field of multimodal single-cell data integration and build a GNNs-based general framework to broadly work on these three key tasks from one NeurIPS competition.
Revised: To the best of our knowledge, we are the first to apply GNNs in the field of multimodal single-cell data integration and build a GNNs-based general framework to broadly work on these three key tasks from one NeurIPS competition.

Original: Our framework officially won first place in the overall ranking of the modality prediction task.
Revised: Our framework officially won first place in the overall ranking of the modality prediction task.

Original: After the competition, we extended our framework to the other two tasks and achieved superior performance compared with the top winning methods.
Revised: After the competition, we extended our framework to the other two tasks and achieved superior performance compared with the top winning methods.

Original:  Problem Statement
Revised:  Problem Statement

Original: There are three modalities spanning through each task, i.e., GEX as mRNA data, ATAC as DNA data and ADT as protein data.
Revised: There are three modalities spanning through each task, i.e., GEX as mRNA data, ATAC as DNA data and ADT as protein data.

Original: Each modality is initially represented by a matrix M ∈ R N × k where N is the number of cells, and k is the feature dimension for each cell.
Revised: Each modality is initially represented by a matrix M ∈ R N × k where N is the number of cells, and k is the feature dimension for each cell.

Original: In our work, we construct a bipartite graph G =
Revised: In our work, we construct a bipartite graph G =

Original: ( U , V , E ) based on each modality M , where U is the set of N cell nodes { u 1 , u 2 , ..., u N } and V is the set of k feature nodes { v 1 , v 2 , ..., v k } .
Revised: ( U , V , E ) based on each modality M , where U is the set of N cell nodes { u 1 , u 2 , ..., u N } and V is the set of k feature nodes { v 1 , v 2 , ..., v k } .

Original: With the aforementioned notations, the problem of learning GNNs for single-cell data integration is formally defined as: given a modality M ∈ R N × k , we aim at learning a mapping function f θ which maps M to the space of downstream tasks.
Revised: With the aforementioned notations, the problem of learning GNNs for single-cell data integration is formally defined as: given a modality M ∈ R N × k , we aim at learning a mapping function f θ which maps M to the space of downstream tasks.

Original: In the following, we formally define these three key tasks of single-cell data integration.
Revised: In the following, we formally define these three key tasks of single-cell data integration.

Original: Task 1: Modality Prediction.
Revised: Task 1: Modality Prediction.

Original: In this task, given one modality (like GEX), the goal is to predict the other (like ATAC) for all feature values in each cell.
Revised: In this task, given one modality (like GEX), the goal is to predict the other (like ATAC) for all feature values in each cell.

Original: It can be formally defined as: given a source modality M 1 ∈ R N × k 1 , the goal is to predict a target modality M 2 ∈ R N × k 2 via learning a mapping function f θ parameterized by θ such that M 

Revised: It can be formally defined as: given a source modality M 1 ∈ R N × k 1 , the goal is to predict a target modality M 2 ∈ R N × k 2 via learning a mapping function f θ parameterized by θ such that M

Original: = f θ ( M 1 ) .
Revised: = f θ ( M 1 ) .

Original: Task 2: Modality Matching.
Revised: Task 2: Modality Matching.

Original: The goal is to identify the correspondence between two single-cell profiles and provide the probability distribution of these predictions.
Revised: The goal is to identify the correspondence between two single-cell profiles and provide the probability distribution of these predictions.

Original: It can be formally defined as: given modality M 1 ∈ R N × k 1 and modality M 2 ∈ R N × k 2 , we aim to learn two mapping functions f θ 1 parameterized by θ 1 and f θ 2 parameterized by θ 2 to map them into the same space such that
Revised: It can be formally defined as: given modality M 1 ∈ R N × k 1 and modality M 2 ∈ R N × k 2 , we aim to learn two mapping functions f θ 1 parameterized by θ 1 and f θ 2 parameterized by θ 2 to map them into the same space such that

Original: [Equation]
Revised: [Equation]

Original: where g is a score function to calculate probability distribution of correspondence predictions.
Revised: where g is a score function to calculate probability distribution of correspondence predictions.

Original: S ∈ R N × N is an output score matrix with each row summing to 1.
Revised: S ∈ R N × N is an output score matrix with each row summing to 1.

Original: S ij is the correspondence probability between i -th cell from modality M 1 and j -th cell from modality M 2 .
Revised: S ij is the correspondence probability between i -th cell from modality M 1 and j -th cell from modality M 2 .

Original: Task 3: Joint Embedding.
Revised: Task 3: Joint Embedding.

Original: In this task, the goal is to learn an embedded representation that leverages the information of two modalities.
Revised: In this task, the goal is to learn an embedded representation that leverages the information of two modalities.

Original: The quality of the embedding will be evaluated using a variety of criteria generated from expert annotation.
Revised: The quality of the embedding will be evaluated using a variety of criteria generated from expert annotation.

Original: It can be formally defined as: given modality M 1 ∈ R N × kand modality M 2 ∈ R N × k 2 , we aim to learn three mapping functions f θ 1 , f θ 2 and f θ 3 parameterized by θ 1 , θ 2 and θ 3 accordingly to project them into downstream tasks,
Revised: It can be formally defined as: given modality M 1 ∈ R N × kand modality M 2 ∈ R N × k 2 , we aim to learn three mapping functions f θ 1 , f θ 2 and f θ 3 parameterized by θ 1 , θ 2 and θ 3 accordingly to project them into downstream tasks,

Original: [Equation]
Revised: [Equation]

Original: where f θ 1 ( M 1 ) ∈ R N × k 1 ′
Revised: where f θ 1 ( M 1 ) ∈ R N × k 1 ′

Original: and f θ 2 ( M 2 ) ∈
Revised: and f θ 2 ( M 2 ) ∈

Original: R N × k 2 ′ correspond to new representations learned from modality M 1 and M 2 separately, H ∈ R N × k 3 is a final output embedding learned through f θ 3 on concatenation of f θ 1 ( M 1 ) and f θ 2 ( M 2 ) .
Revised: R N × k 2 ′ correspond to new representations learned from modality M 1 and M 2 separately, H ∈ R N × k 3 is a final output embedding learned through f θ 3 on concatenation of f θ 1 ( M 1 ) and f θ 2 ( M 2 ) .

Original: H will be measured using six different metrics broken into two classes: biology conservation and batch removal (see Appendix A for more details).
Revised: H will be measured using six different metrics broken into two classes: biology conservation and batch removal (see Appendix A for more details).

Original:  Method 4.1 A General GNN Framework
Revised:  Method 4.1 A General GNN Framework

Original: To develop a GNN-based framework for single cell data integration, there are three challenges: (1) how to construct the graph for cells and its features (or modalities); (2) how to effectively extract meaningful patterns from the graph; and (3) how to adapt the framework to different tasks.
Revised: To develop a GNN-based framework for single cell data integration, there are three challenges: (1) how to construct the graph for cells and its features (or modalities); (2) how to effectively extract meaningful patterns from the graph; and (3) how to adapt the framework to different tasks.

Original: Features
Revised: Features

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: C e ll N od e s 0 … 0 … … … … … …
Revised: C e ll N od e s 0 … 0 … … … … … …

Original: Features
Revised: Features

Original: C e ll s 0 … 0 0 0 … 2 … … … … 2 0 …… …
Revised: C e ll s 0 … 0 0 0 … 2 … … … … 2 0 …… …

Original: Features 0.1 … 0.

Revised: Features 0.1 … 0.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: F ea t u r e s
Revised: F ea t u r e s

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Graph Construction.
Revised: Graph Construction.

Original: With the single-cell data, our first step is to construct a cell-feature graph that GNNs can be applied to.
Revised: With the single-cell data, our first step is to construct a cell-feature graph that GNNs can be applied to.

Original: We construct a cell-feature bipartite graph where the cells and their biological features (e.g. GEX, ADT or ATAC features) are treated as different nodes, which we term as cell nodes and feature nodes, respectively.
Revised: We construct a cell-feature bipartite graph where the cells and their biological features (e.g. GEX, ADT or ATAC features) are treated as different nodes, which we term as cell nodes and feature nodes, respectively.

Original: A cell node is connected with the feature nodes that represent its features.
Revised: A cell node is connected with the feature nodes that represent its features.

Original: With such graph structure, the advantage is that cell nodes can propagate features to their neighboring feature nodes, and vice versa.
Revised: With such graph structure, the advantage is that cell nodes can propagate features to their neighboring feature nodes, and vice versa.

Original: Formally, we denote the bipartite graph as G =
Revised: Formally, we denote the bipartite graph as G =

Original: ( U , V , E ) .
Revised: ( U , V , E ) .

Original: In this graph, U is the set of N cell nodes { u 1 , u 2 , ..., u N } and V is the set of k feature nodes { v 1 , v 2 , ..., v k } , where each feature node refers to one feature dimension of the input data.
Revised: In this graph, U is the set of N cell nodes { u 1 , u 2 , ..., u N } and V is the set of k feature nodes { v 1 , v 2 , ..., v k } , where each feature node refers to one feature dimension of the input data.

Original: E ⊆
Revised: E ⊆

Original: U × V represents the set of edges between U and
Revised: U × V represents the set of edges between U and

Original: V , which describe the relations between cells and features.
Revised: V , which describe the relations between cells and features.

Original: The graph can be denoted as a weighted adjacency matrix A =
Revised: The graph can be denoted as a weighted adjacency matrix A =

Original: [Equation]
Revised: [Equation]

Original: ) , where O is a matrix filled with constant 0 and M ∈ R N × k is the input feature matrix of cells.
Revised: ) , where O is a matrix filled with constant 0 and M ∈ R N × k is the input feature matrix of cells.

Original: M can be also viewed as the features of one modality such as GEX.
Revised: M can be also viewed as the features of one modality such as GEX.

Original: Note that A is a bipartite graph where two nodes within the same set (either feature nodes or cell nodes) are not adjacent.
Revised: Note that A is a bipartite graph where two nodes within the same set (either feature nodes or cell nodes) are not adjacent.

Original: Based on the aforementioned process of graph construction, we can adjust it for specific tasks, e.g., incorporating a prior knowledge graph of genes, which can change the bipartite characteristic of A .
Revised: Based on the aforementioned process of graph construction, we can adjust it for specific tasks, e.g., incorporating a prior knowledge graph of genes, which can change the bipartite characteristic of A .

Original: Furthermore, since GNNs are mostly dealing with attributed graphs, we need to assign initial embeddings for feature and cell nodes.
Revised: Furthermore, since GNNs are mostly dealing with attributed graphs, we need to assign initial embeddings for feature and cell nodes.

Original: Specifically, we use X cell and X feat to denote the initial embeddings for cell and feature nodes, respectively.
Revised: Specifically, we use X cell and X feat to denote the initial embeddings for cell and feature nodes, respectively.

Original: We have
Revised: We have

Original: X cell ∈ R N × d ′ and X feat ∈ R k × d ′′ where d ′ and d ′′ are determined by the task-specific settings.
Revised: X cell ∈ R N × d ′ and X feat ∈ R k × d ′′ where d ′ and d ′′ are determined by the task-specific settings.

Original: As an illustrative example, the initial embeddings of feature nodes { v 1 , ..., v k } could be the one-hot index of each feature dimension; thus, X feat ∈
Revised: As an illustrative example, the initial embeddings of feature nodes { v 1 , ..., v k } could be the one-hot index of each feature dimension; thus, X feat ∈

Original: R k × k is an identity matrix, i.e., X feat
Revised: R k × k is an identity matrix, i.e., X feat

Original: =
Revised: =

Original: I k .
Revised: I k .

Original: Meanwhile, we do not have any prior knowledge for each cell, thus, X cell =
Revised: Meanwhile, we do not have any prior knowledge for each cell, thus, X cell =

Original: O N × 1 .
Revised: O N × 1 .

Original: Together with the node features, the constructed cell-feature graph can be denoted as G =
Revised: Together with the node features, the constructed cell-feature graph can be denoted as G =

Original: ( A , X feat , X cell ) .
Revised: ( A , X feat , X cell ) .

Original: Cell-Feature Graph Convolution.
Revised: Cell-Feature Graph Convolution.

Original: After we obtain the constructed cell-feature graph, the next challenge is how to extract high-order structural information from the graph.
Revised: After we obtain the constructed cell-feature graph, the next challenge is how to extract high-order structural information from the graph.

Original: In this work, we utilize GNNs to effectively learn cell/feature representations over the constructed graph.
Revised: In this work, we utilize GNNs to effectively learn cell/feature representations over the constructed graph.

Original: Note that there are two types of nodes in the graph and we need to deal with them differently.
Revised: Note that there are two types of nodes in the graph and we need to deal with them differently.

Original: For the ease of illustration, we first only consider one type of nodes (e.g., feature nodes V ), and later we extend it to the whole graph.
Revised: For the ease of illustration, we first only consider one type of nodes (e.g., feature nodes V ), and later we extend it to the whole graph.

Original: Typically, GNNs follow the message-passing paradigm [11] and in each layer of GNNs, the embedding of each node is updated according to the messages passed from its neighbors.
Revised: Typically, GNNs follow the message-passing paradigm [11] and in each layer of GNNs, the embedding of each node is updated according to the messages passed from its neighbors.

Original: Let H l
Revised: Let H l

Original: = { h l 1 , ..., h lN } , h li ∈ R dl be the input node embeddings in the l -th layer, where h li corresponds to node v i .
Revised: = { h l 1 , ..., h lN } , h li ∈ R dl be the input node embeddings in the l -th layer, where h li corresponds to node v i .

Original: Hence, the output embeddings of the l -th layer can be expressed as h l +

Revised: Hence, the output embeddings of the l -th layer can be expressed as h l +

Original: i
Revised: i

Original: =
Revised: =

Original: Update( h l i , Agg( h l j | j ∈ N i )) , where N i is the set of first-order neighbors of node v i , Agg( · ) indicates an aggregation function on neighbor nodes’ embedding, and Update( · ) is an update function that generates a new node embedding vector from the previous one and aggregation results from neighbors.
Revised: Update( h l i , Agg( h l j | j ∈ N i )) , where N i is the set of first-order neighbors of node v i , Agg( · ) indicates an aggregation function on neighbor nodes’ embedding, and Update( · ) is an update function that generates a new node embedding vector from the previous one and aggregation results from neighbors.

Original: Notably, for the input node embedding in the first layer, we have
Revised: Notably, for the input node embedding in the first layer, we have

Original: H 1 = σ ( X feat W feat + b feat ) , where W feat ∈ R d ′ ×
Revised: H 1 = σ ( X feat W feat + b feat ) , where W feat ∈ R d ′ ×

Original: d is a transformation matrix, d is the dimension of the hidden space and σ is an activation function.
Revised: d is a transformation matrix, d is the dimension of the hidden space and σ is an activation function.

Original: Though there exist a number of different GNN models, in this work, we focus on the most representative one, Graph Convolution Network (GCN) [14].
Revised: Though there exist a number of different GNN models, in this work, we focus on the most representative one, Graph Convolution Network (GCN) [14].

Original: Note that it is straightforward to extend the proposed framework to other GNN models.
Revised: Note that it is straightforward to extend the proposed framework to other GNN models.

Original: Considering that we have two different types of nodes in the graph (i.e., cells and features nodes), we make some modifications on the vanilla GCN to deal with different types of nodes/edges.
Revised: Considering that we have two different types of nodes in the graph (i.e., cells and features nodes), we make some modifications on the vanilla GCN to deal with different types of nodes/edges.

Original: We name it as cell-feature graph convolution , where we separately perform the aggregation function on different types of edges to capture interactions between cell and feature nodes.
Revised: We name it as cell-feature graph convolution , where we separately perform the aggregation function on different types of edges to capture interactions between cell and feature nodes.

Original: Moreover, we use different parameters for aggregation on different edges, thus allowing the embedding of each node type to have very different distributions.
Revised: Moreover, we use different parameters for aggregation on different edges, thus allowing the embedding of each node type to have very different distributions.

Original: Specifically, from a message passing perspective, the operation in a cell-feature graph convolution layer can be expressed as two steps, i.e., aggregation and updating.
Revised: Specifically, from a message passing perspective, the operation in a cell-feature graph convolution layer can be expressed as two steps, i.e., aggregation and updating.

Original: In order to generate a message m for different types of nodes, there are at least two basic aggregation functions, one is:
Revised: In order to generate a message m for different types of nodes, there are at least two basic aggregation functions, one is:

Original: [Equation]
Revised: [Equation]

Original: where i corresponds to node v i ∈ V , j corresponds to node u j ∈ U ;
Revised: where i corresponds to node v i ∈ V , j corresponds to node u j ∈ U ;

Original: e ji denotes the edge weight between v j and u i , W l U→V and b l U→V are trainable parameters, σ ( · ) is an activation function such as ReLU, and c ji is a normalization term defined as c ji = (cid:113)(cid:80) k ∈N j e jk
Revised: e ji denotes the edge weight between v j and u i , W l U→V and b l U→V are trainable parameters, σ ( · ) is an activation function such as ReLU, and c ji is a normalization term defined as c ji = (cid:113)(cid:80) k ∈N j e jk

Original: (cid:113)(cid:80) k ∈N i e ki .
Revised: (cid:113)(cid:80) k ∈N i e ki .

Original: Obviously,
Revised: Obviously,

Original: Eq.
Revised: Eq.

Original: (3) is the aggregation function from cell nodes U to feature nodes V .
Revised: (3) is the aggregation function from cell nodes U to feature nodes V .

Original: Thus the other aggregation function from V to U can be written as:
Revised: Thus the other aggregation function from V to U can be written as:

Original: [Equation]
Revised: [Equation]

Original: The transformation matrices W l U→V and W l V→U project the node embeddings from one hidden space to another vice versa.
Revised: The transformation matrices W l U→V and W l V→U project the node embeddings from one hidden space to another vice versa.

Original: After generating the messages from neighborhoods, we then update the embedding for nodes in V and U accordingly:
Revised: After generating the messages from neighborhoods, we then update the embedding for nodes in V and U accordingly:

Original: [Equation]
Revised: [Equation]

Original: where v i ∈ V and u j ∈ U .
Revised: where v i ∈ V and u j ∈ U .

Original: In Eq.
Revised: In Eq.

Original: (5), we adopt a simple residual mechanism in order to enhance self information.
Revised: (5), we adopt a simple residual mechanism in order to enhance self information.

Original: As mentioned earlier, we can have more than two types of edges depending on the downstream task and the graph structure can be much more complex than a bipartite graph.
Revised: As mentioned earlier, we can have more than two types of edges depending on the downstream task and the graph structure can be much more complex than a bipartite graph.

Original: Despite such complexity, our proposed framework and cell-feature graph convolution have the capacity to handle more types of edges/nodes.
Revised: Despite such complexity, our proposed framework and cell-feature graph convolution have the capacity to handle more types of edges/nodes.

Original: We will introduce these details in Section 4.2.
Revised: We will introduce these details in Section 4.2.

Original: Task-specific Head.
Revised: Task-specific Head.

Original: After we learn node embeddings for feature and cell nodes, we need to project the embedding to the space of the specific downstream task.
Revised: After we learn node embeddings for feature and cell nodes, we need to project the embedding to the space of the specific downstream task.

Original: Hence, we design a task-specific head, which depends on the downstream task.
Revised: Hence, we design a task-specific head, which depends on the downstream task.

Original: Specifically, we first take the node embeddings of cell nodes from each convolution layer, aggregate them and project them into the space of downstream task ˆY : ˆY =
Revised: Specifically, we first take the node embeddings of cell nodes from each convolution layer, aggregate them and project them into the space of downstream task ˆY : ˆY =

Original: Head (cid:0) Readout θ
Revised: Head (cid:0) Readout θ

Original: ( H 1 U , ..., H L U ) (cid:1) , where H i U refers to embeddings of all cell nodes in i -th layer,
Revised: ( H 1 U , ..., H L U ) (cid:1) , where H i U refers to embeddings of all cell nodes in i -th layer,

Original: Readout( · ) θ is a trainable aggregation function, Head( · ) is a linear transformation that projects the latent embedding to the downstream task space.
Revised: Readout( · ) θ is a trainable aggregation function, Head( · ) is a linear transformation that projects the latent embedding to the downstream task space.

Original: With the obtained output, we can then optimize the framework through minimizing the task-specific loss functions.
Revised: With the obtained output, we can then optimize the framework through minimizing the task-specific loss functions.

Original: In the following subsection, we give the details of training the general framework for different tasks.
Revised: In the following subsection, we give the details of training the general framework for different tasks.

Original: 4.2 Model Specifications
Revised: 4.2 Model Specifications

Original: We show how scMoGNN is applied in the three important tasks in single-cell data integration.
Revised: We show how scMoGNN is applied in the three important tasks in single-cell data integration.

Original: Modality Prediction.
Revised: Modality Prediction.

Original: In the modality prediction task, our objective is to translate the data from one modality to another.
Revised: In the modality prediction task, our objective is to translate the data from one modality to another.

Original: Given the flexibility of graph construction and GNNs in our framework, we can readily incorporate external knowledge into our method.
Revised: Given the flexibility of graph construction and GNNs in our framework, we can readily incorporate external knowledge into our method.

Original: In this task, we adjust the graph construction to include such domain knowledge to enhance the feature information.
Revised: In this task, we adjust the graph construction to include such domain knowledge to enhance the feature information.

Original: Intuitively, we can construct a fully-connected inter-gene graph for each gene set, and incorporate those edges into our original graph G .
Revised: Intuitively, we can construct a fully-connected inter-gene graph for each gene set, and incorporate those edges into our original graph G .

Original: Hence, the new adjacency matrix for G is A = (cid:18)
Revised: Hence, the new adjacency matrix for G is A = (cid:18)

Original: O
Revised: O

Original: M
Revised: M

Original: M T
Revised: M T

Original: P (cid:19) , where P ∈ R k × k is a symmetric matrix, which refers to the links between gene features, generated from gene sets data.
Revised: P (cid:19) , where P ∈ R k × k is a symmetric matrix, which refers to the links between gene features, generated from gene sets data.

Original: Since cell and feature nodes have very different characteristics, when we aggregate the embedding from N u
Revised: Since cell and feature nodes have very different characteristics, when we aggregate the embedding from N u

Original: i and N v i respectively, we expect to get very different results.
Revised: i and N v i respectively, we expect to get very different results.

Original: Thus, when we update the embedding of center node, we have different strategies to combine messages from different channels.
Revised: Thus, when we update the embedding of center node, we have different strategies to combine messages from different channels.

Original: As a starting point, we decide to enable a scalar weight.
Revised: As a starting point, we decide to enable a scalar weight.

Original: Formally speaking, similar to Eq.
Revised: Formally speaking, similar to Eq.

Original: (3) and
Revised: (3) and

Original: Eq.
Revised: Eq.

Original: (4), we first generate two messages
Revised: (4), we first generate two messages

Original: m i,l U→V and m i,l V→V for each node v i ∈ V , then we update the node embedding of v i following h l
Revised: m i,l U→V and m i,l V→V for each node v i ∈ V , then we update the node embedding of v i following h l

Original: +

Revised: +

Original: i
Revised: i

Original: =
Revised: =

Original: h li + α · m i,l V→V + (1 − α ) ·
Revised: h li + α · m i,l V→V + (1 − α ) ·

Original: m i,l U→V , where α is either a hyper-parameter or a learnable scaler to control the ratio between inter-feature aggregation and cell-feature aggregation.
Revised: m i,l U→V , where α is either a hyper-parameter or a learnable scaler to control the ratio between inter-feature aggregation and cell-feature aggregation.

Original: The head structure for modality prediction is relatively simple.
Revised: The head structure for modality prediction is relatively simple.

Original: Note that we deliberately keep the same hidden dimension throughout the cell-feature graph convolution; thus we can simply use a trainable weight vector w to sum up cell node embeddings from different layers:
Revised: Note that we deliberately keep the same hidden dimension throughout the cell-feature graph convolution; thus we can simply use a trainable weight vector w to sum up cell node embeddings from different layers:

Original: ˆH = (cid:80) Li
Revised: ˆH = (cid:80) Li

Original: =
Revised: =

Original: 1 w i · H i U , where ˆH , H i U ∈
Revised: 1 w i · H i U , where ˆH , H i U ∈

Original: R N × d , and d is the dimension of hidden layer.
Revised: R N × d , and d is the dimension of hidden layer.

Original: After that, a simple fully connected layer is performed to transform it to the target space: ˆY = ˆHW + b .
Revised: After that, a simple fully connected layer is performed to transform it to the target space: ˆY = ˆHW + b .

Original: A rooted mean squared error (RMSE) loss is then calculated as L = (cid:113) 1 N (cid:80) Ni
Revised: A rooted mean squared error (RMSE) loss is then calculated as L = (cid:113) 1 N (cid:80) Ni

Original: =

Revised: =

Original: ( Y i − ˆY i ) 2 .
Revised: ( Y i − ˆY i ) 2 .

Original: Modality Matching.
Revised: Modality Matching.

Original: Our goal in the modality matching task is to predict the probability that a pair of different modality data is actually from the same cell.
Revised: Our goal in the modality matching task is to predict the probability that a pair of different modality data is actually from the same cell.

Original: Since two different modalities are presented as input in this task, we construct two cell-feature graphs.
Revised: Since two different modalities are presented as input in this task, we construct two cell-feature graphs.

Original: Cell-feature graph convolutions are then performed on these two graphs separately to obtain two embedding matrices ˆH m1 ∈ R N × d 1 and ˆH m2 ∈ R N × d 2 .
Revised: Cell-feature graph convolutions are then performed on these two graphs separately to obtain two embedding matrices ˆH m1 ∈ R N × d 1 and ˆH m2 ∈ R N × d 2 .

Original: We set d 1 = d 2 so they can be directly multiplied together.
Revised: We set d 1 = d 2 so they can be directly multiplied together.

Original: Thus, we calculate the cosine similarity between each cell pair as S = ˆH ′ m1 · ˆH ′ T m2 , where S ∈ R N × N denotes the symmetric score matrix; ˆH ′ m1 and ˆH ′ m2 indicate that we perform L2 normalization for each row (i.e., each cell) in ˆH m1 and ˆH m2 before we perform matrix multiplication.
Revised: Thus, we calculate the cosine similarity between each cell pair as S = ˆH ′ m1 · ˆH ′ T m2 , where S ∈ R N × N denotes the symmetric score matrix; ˆH ′ m1 and ˆH ′ m2 indicate that we perform L2 normalization for each row (i.e., each cell) in ˆH m1 and ˆH m2 before we perform matrix multiplication.

Original: Then we can express the loss function as follows:
Revised: Then we can express the loss function as follows:

Original: [Equation]
Revised: [Equation]

Original: c 1 ,c 2 log( P rc 1 ,c 2 )
Revised: c 1 ,c 2 log( P rc 1 ,c 2 )

Original: + Y c 1 ,c 2 log( P cc 1 ,c 2 ) with P ri,j = e S i,j (cid:80)
Revised: + Y c 1 ,c 2 log( P cc 1 ,c 2 ) with P ri,j = e S i,j (cid:80)

Original: Nk
Revised: Nk

Original: [Equation]
Revised: [Equation]

Original: = e
Revised: = e

Original: [Equation]
Revised: [Equation]

Original: where Y ∈ R N × N denotes a binarized matching matrix that indicates the perfect matching (i.e., the ground truth label), and P i,j ∈ R N × N represents the probability that i -th data in modality 1 and j -th data in modality 2 are actually referring to the same cell.
Revised: where Y ∈ R N × N denotes a binarized matching matrix that indicates the perfect matching (i.e., the ground truth label), and P i,j ∈ R N × N represents the probability that i -th data in modality 1 and j -th data in modality 2 are actually referring to the same cell.

Original: In addition to L match , we include a set of auxiliary losses to boost the performance, i.e., prediction losses and reconstruction losses:
Revised: In addition to L match , we include a set of auxiliary losses to boost the performance, i.e., prediction losses and reconstruction losses:

Original: [Equation]
Revised: [Equation]

Original: where X m1 and X m2 refer to the preprocessing results of two modalities respectively, f θ 1 and f θeach refers to a fully connected network with one hidden layer, which project the node embeddings ˆH to the particular target modality space.
Revised: where X m1 and X m2 refer to the preprocessing results of two modalities respectively, f θ 1 and f θeach refers to a fully connected network with one hidden layer, which project the node embeddings ˆH to the particular target modality space.

Original: These auxiliary losses provide extra supervision for our model to encode the necessary information within the hidden space ˆH .
Revised: These auxiliary losses provide extra supervision for our model to encode the necessary information within the hidden space ˆH .

Original: Hence they have the potential to improve the robustness of the model and reduce the risk of overfitting.
Revised: Hence they have the potential to improve the robustness of the model and reduce the risk of overfitting.

Original: In the training phase, we jointly optimize L match and L aux .
Revised: In the training phase, we jointly optimize L match and L aux .

Original: Lastly, in the inference phase, we introduce bipartite matching as an extra post-processing method to further augment our matching result.
Revised: Lastly, in the inference phase, we introduce bipartite matching as an extra post-processing method to further augment our matching result.

Original: Joint Embedding.
Revised: Joint Embedding.

Original: The target of the joint embedding task is to learn cell embeddings from multiple modalities and thus better describe cellular heterogeneity.
Revised: The target of the joint embedding task is to learn cell embeddings from multiple modalities and thus better describe cellular heterogeneity.

Original: Several complex metrics are enabled in this task, there often exist trade-offs between metrics and metrics (e.g. to remove the batch effect while retaining the batch information).
Revised: Several complex metrics are enabled in this task, there often exist trade-offs between metrics and metrics (e.g. to remove the batch effect while retaining the batch information).

Original: To provide more training signals, we utilize both supervised and self-supervised losses to train our graph neural networks.
Revised: To provide more training signals, we utilize both supervised and self-supervised losses to train our graph neural networks.

Original: Specifically, we first use the LSI for preprocessing to generate the input node features for two modalities and concatenate them as one joint modality, which allows us to construct a cell-feature graph.
Revised: Specifically, we first use the LSI for preprocessing to generate the input node features for two modalities and concatenate them as one joint modality, which allows us to construct a cell-feature graph.

Original: Based on the graph, we perform the proposed cell-feature graph convolution and generate the output cell node embedding in the same way as in modality prediction.
Revised: Based on the graph, we perform the proposed cell-feature graph convolution and generate the output cell node embedding in the same way as in modality prediction.

Original: As suggested by the work [16], cell type information plays a key role in the metrics of joint embedding evaluation and it is beneficial to extract T dimensions from the hidden space to serve as supervision signals.
Revised: As suggested by the work [16], cell type information plays a key role in the metrics of joint embedding evaluation and it is beneficial to extract T dimensions from the hidden space to serve as supervision signals.

Original: Following this idea, we calculate the softmax function for t ∈ { 1 , ..., T } , i ∈ { 1 , ..., N } with N being the number of cells:
Revised: Following this idea, we calculate the softmax function for t ∈ { 1 , ..., T } , i ∈ { 1 , ..., N } with N being the number of cells:

Original: ˆY i
Revised: ˆY i

Original: [Equation]
Revised: [Equation]

Original: t =
Revised: t =

Original: [Equation]
Revised: [Equation]

Original: , where ˆY is the probability that cell i belongs to cell type t and T is set to be exactly equal to the total number of cell types.
Revised: , where ˆY is the probability that cell i belongs to cell type t and T is set to be exactly equal to the total number of cell types.

Original: Then we introduce the loss functions:
Revised: Then we introduce the loss functions:

Original: [Equation]
Revised: [Equation]

Original: where f θ is a two-layer perceptron, Y ∈ R N × T is the sparse labels of cell types, and ˆH ˜ J refers to the other hidden dimensions aside from the T dimensions that have been exclusive to cell type
Revised: where f θ is a two-layer perceptron, Y ∈ R N × T is the sparse labels of cell types, and ˆH ˜ J refers to the other hidden dimensions aside from the T dimensions that have been exclusive to cell type

Original: [Table]
Revised: [Table]

Original: information.
Revised: information.

Original: Eventually, the output hidden space ˆH would contain cellular information required by the task, although the loss functions do not directly optimize the final metric
Revised: Eventually, the output hidden space ˆH would contain cellular information required by the task, although the loss functions do not directly optimize the final metric

Original: [Table]
Revised: [Table]

Original:  Experiment
Revised:  Experiment

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we evaluate the effectiveness of our framework scMoGNN against three tasks and show how scMoGNN outperforms representative baselines over all three tasks by combining our general framework with task-specific design.
Revised: we evaluate the effectiveness of our framework scMoGNN against three tasks and show how scMoGNN outperforms representative baselines over all three tasks by combining our general framework with task-specific design.

Revised: All source codes of our model have been integrated into the DANCE package [8].

Original: We defer the experimental setup to Appendix.
Revised: We defer the experimental setup to Appendix

Revised: [Table]

Original: 5.1 Main Results
Revised: 5.1 Main Results

Original: Modality Prediction.
Revised: Modality Prediction.

Original: As shown in Table 1, our method achieved the lowest overall loss in the competition (the lower, the better).
Revised: As shown in Table 1, our method achieved the lowest overall loss in the competition (the lower, the better).

Original: An interesting observation is that there is no team lead consistently across all subtasks, resulting in individual s for each category, which is very different from the other two tasks in the competition.
Revised: An interesting observation is that there is no team lead consistently across all subtasks, resulting in individual s for each category, which is very different from the other two tasks in the competition.

Original: However, as far as we know, there is no team that worked only on one subtask.
Revised: However, as far as we know, there is no team that worked only on one subtask.

Original: Such a phenomenon may be caused by three reasons: (1) the modality prediction task is the most competitive task in the competition, and many participating teams participated only in this task (including our own team).
Revised: Such a phenomenon may be caused by three reasons: (1) the modality prediction task is the most competitive task in the competition, and many participating teams participated only in this task (including our own team).

Original: As a result, over 40 individual teams appeared on the final leaderboard.
Revised: As a result, over 40 individual teams appeared on the final leaderboard.

Original: (2) the modality prediction task has only 1,000 cells in the private test dataset, therefore, certain variance exists in the evaluation results.
Revised: (2) the modality prediction task has only 1,000 cells in the private test dataset, therefore, certain variance exists in the evaluation results.

Original: (3) the diverse feature dimensionality and sparsity in different modalities raised additional challenges to the model’s generalization ability.
Revised: (3) the diverse feature dimensionality and sparsity in different modalities raised additional challenges to the model’s generalization ability.

Original: Compared to the other models, our GNN model presented consistently better performance across these four subtasks and became the overall winner in the competition.
Revised: Compared to the other models, our GNN model presented consistently better performance across these four subtasks and became the overall winner in the competition.

Original: Furthermore, we even improved our models after the competition, with modifications including: a learning-rate decay training strategy, more hidden units along with stronger regularization of dropout and weight decay.
Revised: Furthermore, we even improved our models after the competition, with modifications including: a learning-rate decay training strategy, more hidden units along with stronger regularization of dropout and weight decay.

Original: Eventually, we’ve effectively strengthened our graph neural network model hence significantly improved our results, especially in the toughest subtask GEX-to-ADT, where the output for each cell is a 134-dimensional dense vector.
Revised: Eventually, we’ve effectively strengthened our graph neural network model hence significantly improved our results, especially in the toughest subtask GEX-to-ADT, where the output for each cell is a 134-dimensional dense vector.

Original: We now achieved an RMSE loss of 0 . 3809 which is lower than the previous best score of 0 . 3854 in the competition.
Revised: We now achieved an RMSE loss of 0 . 3809 which is lower than the previous best score of 0 . 3854 in the competition.

Original: Overall, the results prove the effectiveness of our GNN framework, and in some specific cases, scMoGNN has tremendous advantage in view of performance.
Revised: Overall, the results prove the effectiveness of our GNN framework, and in some specific cases, scMoGNN has tremendous advantage in view of performance.

Original: Modality Matching.
Revised: Modality Matching.

Original: As shown in Table 2, scMoGNN outperforms the winning team and the runnerup team with a very large margin.
Revised: As shown in Table 2, scMoGNN outperforms the winning team and the runnerup team with a very large margin.

Original: Note that we didn’t create any models for this task during the competition since we focused on the modality prediction task.
Revised: Note that we didn’t create any models for this task during the competition since we focused on the modality prediction task.

Original: This is the reason why we don’t have any results on the official leaderboard.
Revised: This is the reason why we don’t have any results on the official leaderboard.

Original: The score of the metric can be roughly seen as the accuracy of predicting a right correspondence for each piece of data.
Revised: The score of the metric can be roughly seen as the accuracy of predicting a right correspondence for each piece of data.

Original: Meanwhile the search space grows with the total number of cells in the test data.
Revised: Meanwhile the search space grows with the total number of cells in the test data.

Original: For example, in the test phase of the ADTto-GEX subtask, we have 15,066 cells to match, thus for each piece of ADT data, we have 15, candidates in GEX data.
Revised: For example, in the test phase of the ADTto-GEX subtask, we have 15,066 cells to match, thus for each piece of ADT data, we have 15, candidates in GEX data.

Original: Although we separated those cells into three batches, the rough expectation of the accuracy of randomly guessing is still as low as 1/5000, which can indicate the difficulty of this task.
Revised: Although we separated those cells into three batches, the rough expectation of the accuracy of randomly guessing is still as low as 1/5000, which can indicate the difficulty of this task.

Original: Thus, scMoGNN has already achieved very high performance (e.g. 0.08 in ADT-GEX subtask).
Revised: Thus, scMoGNN has already achieved very high performance (e.g. 0.08 in ADT-GEX subtask).

Original: Note that both team Novel’s model and scMoGNN utilizes a symmetric matching algorithm, thus we have exactly the same performance for dual subtasks (e.g. GEX2ADT and ADT2GEX).
Revised: Note that both team Novel’s model and scMoGNN utilizes a symmetric matching algorithm, thus we have exactly the same performance for dual subtasks (e.g. GEX2ADT and ADT2GEX).

Original: Another interesting observation is that our proposed graph neural network model is especially good at
Revised: Another interesting observation is that our proposed graph neural network model is especially good at

Original: GEX-ADT dual subtasks, where we improved the previous winning performance from 0 .
Revised: GEX-ADT dual subtasks, where we improved the previous winning performance from 0 .

Original: 05 to 0 .
Revised: 05 to 0 .

Original: Joint Embedding.
Revised: Joint Embedding.

Original: In Table 3, our scMoGNN significantly outperforms the other two models in
Revised: In Table 3, our scMoGNN significantly outperforms the other two models in

Original: GEX-ADT joint embedding task, with an improvement over 0.1 according to the average metric.
Revised: GEX-ADT joint embedding task, with an improvement over 0.1 according to the average metric.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: T a s k T a s k
Revised: T a s k T a s k

Original: T a s k
Revised: T a s k

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 5.2 Further Analysis
Revised: 5.2 Further Analysis

Original: Ablation Study.
Revised: Ablation Study.

Original: We investigate if we really benefit from graph structural information.
Revised: We investigate if we really benefit from graph structural information.

Original: We take the modality matching task as an example.
Revised: We take the modality matching task as an example.

Original: In the modality matching task we use decoupled GNNs, thus, we can easily remove the graph structural information by eliminating the propagation layers.
Revised: In the modality matching task we use decoupled GNNs, thus, we can easily remove the graph structural information by eliminating the propagation layers.

Original: The result is referred to as “w/o propagation” in Figure 3.
Revised: The result is referred to as “w/o propagation” in Figure 3.

Original: The performance significantly drops from 0 .
Revised: The performance significantly drops from 0 .

Original: 0810 to 0 .
Revised: 0810 to 0 .

Original: 0745 in the GEX-ADT subtask and from 0 .
Revised: 0745 in the GEX-ADT subtask and from 0 .

Original: 0630 to 0 . 0562 in the GEX-ATAC subtask, respectively.
Revised: 0630 to 0 . 0562 in the GEX-ATAC subtask, respectively.

Original: These observations indicate that the graph structural information extracted by the propagation layers indeed helped the performance of our method significantly.
Revised: These observations indicate that the graph structural information extracted by the propagation layers indeed helped the performance of our method significantly.

Original: We also examined the importance of our auxiliary loss, shown in Figure 3.
Revised: We also examined the importance of our auxiliary loss, shown in Figure 3.

Original: Without the supervision of auxiliary losses, scMoGNN lost generalization ability, behaving as poorly as without graph structural information.
Revised: Without the supervision of auxiliary losses, scMoGNN lost generalization ability, behaving as poorly as without graph structural information.

Original: Parameter Analysis.
Revised: Parameter Analysis.

Original: We analyzed an important parameter in our framework, i.e. w in ˆH = (cid:80)
Revised: We analyzed an important parameter in our framework, i.e. w in ˆH = (cid:80)

Original: Li =1 w i · H i U in modality prediction, to gain a deeper insight on scMoGNN .
Revised: Li =1 w i · H i U in modality prediction, to gain a deeper insight on scMoGNN .

Original: We show values of w learned by scMoGNN in different tasks in Figure 2.
Revised: We show values of w learned by scMoGNN in different tasks in Figure 2.

Original: Note that in different tasks we have different numbers of layers, in modality prediction we have 4 layers and in modality matching and joint embedding we have 3 layers and 2 layers, respectively.
Revised: Note that in different tasks we have different numbers of layers, in modality prediction we have 4 layers and in modality matching and joint embedding we have 3 layers and 2 layers, respectively.

Original: The results consistently show that scMoGNN tends to synthesize the information in each layer, not just limited to the shallow layer, which suggests that the information of the higher-order graph structure is indeed effective.
Revised: The results consistently show that scMoGNN tends to synthesize the information in each layer, not just limited to the shallow layer, which suggests that the information of the higher-order graph structure is indeed effective.

Original: As for more details, joint embedding depends more on local information, which exists in source input data.
Revised: As for more details, joint embedding depends more on local information, which exists in source input data.

Original: In modality prediction, more higher-order information is referenced, indicating that the model needs to enrich more information from similar cells or similar features.
Revised: In modality prediction, more higher-order information is referenced, indicating that the model needs to enrich more information from similar cells or similar features.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we proposed a general framework scMoGNN based on GNNs for multimodal single-cell data integration.
Revised: In this work, we proposed a general framework scMoGNN based on GNNs for multimodal single-cell data integration.

Original: It can be broadly applied on all three key tasks, modality prediction, modality matching and joint embedding.
Revised: It can be broadly applied on all three key tasks, modality prediction, modality matching and joint embedding.

Original: Our framework scMoGNN is able to capture high-order structural information between cells and features.
Revised: Our framework scMoGNN is able to capture high-order structural information between cells and features.

Original: To the best of our knowledge, we are the first to apply GNNs in this field.
Revised: To the best of our knowledge, we are the first to apply GNNs in this field.

Original: Our method officially wins first place in the overall ranking of the modality prediction task and now outperforms all models from three tasks on the leaderboard with remarkable advantage.
Revised: Our method officially wins first place in the overall ranking of the modality prediction task and now outperforms all models from three tasks on the leaderboard with remarkable advantage.

Original: A Details of Metrics in TaskA.1 Biology Conservation Metrics
Revised: A Details of Metrics in TaskA.1 Biology Conservation Metrics

Original: 1. NMI cluster/label:
Revised: 1. NMI cluster/label:

Original: Normalized mutual information (NMI) compares the overlap of two clusterings.
Revised: Normalized mutual information (NMI) compares the overlap of two clusterings.

Original: We use NMI to compare the cell type labels with an automated clustering computed on the integrated dataset (based on Louvain clustering 5 ).
Revised: We use NMI to compare the cell type labels with an automated clustering computed on the integrated dataset (based on Louvain clustering 5 ).

Original: NMI scores of 0 or 1 correspond to uncorrelated clustering or a perfect match, respectively.
Revised: NMI scores of 0 or 1 correspond to uncorrelated clustering or a perfect match, respectively.

Original: Automated Louvain clustering is performed at resolution ranges from 0.1 to 2 in steps of 0.1, and the clustering output with the highest NMI with the label set is used.
Revised: Automated Louvain clustering is performed at resolution ranges from 0.1 to 2 in steps of 0.1, and the clustering output with the highest NMI with the label set is used.

Original: Cell type ASW:
Revised: Cell type ASW:

Original: The silhouette width metric indicates the degree to which observations with identical labels are compact.
Revised: The silhouette width metric indicates the degree to which observations with identical labels are compact.

Original: The average silhouette width (ASW) [2], which ranges betweenand 1, is calculated by averaging the silhouette widths of all cells in a set.
Revised: The average silhouette width (ASW) [2], which ranges betweenand 1, is calculated by averaging the silhouette widths of all cells in a set.

Original: We employ ASW to determine the compactness of the resulting embedding’s cell types.
Revised: We employ ASW to determine the compactness of the resulting embedding’s cell types.

Original: The ASW of the cluster is calculated using the cell identity labels and scaled to a value between 0 and 1 using the equation:
Revised: The ASW of the cluster is calculated using the cell identity labels and scaled to a value between 0 and 1 using the equation:

Original: [Equation]
Revised: [Equation]

Original: where C denotes the set of all cell identity labels.
Revised: where C denotes the set of all cell identity labels.

Original: Cell cycle conservation: The cell cycle conservation score serves as a proxy for the preservation of the signal associated with gene programs during data integration.
Revised: Cell cycle conservation: The cell cycle conservation score serves as a proxy for the preservation of the signal associated with gene programs during data integration.

Original: It determines the amount of variance explained by cell cycle per batch prior to and following integration.
Revised: It determines the amount of variance explained by cell cycle per batch prior to and following integration.

Original: The differences in variance before V ar before and variance after V ar after are aggregated into a final score between 0 and 1, using the equation:
Revised: The differences in variance before V ar before and variance after V ar after are aggregated into a final score between 0 and 1, using the equation:

Original: [Equation]
Revised: [Equation]

Original: where values near to 0 suggest less conservation of variance explained by the cell cycle, whilerepresents complete conservation.
Revised: where values near to 0 suggest less conservation of variance explained by the cell cycle, whilerepresents complete conservation.

Original: Trajectory conservation: The conservation score of a trajectory is a proxy for the conservation of a continuous biological signal within a joint embedding.
Revised: Trajectory conservation: The conservation score of a trajectory is a proxy for the conservation of a continuous biological signal within a joint embedding.

Original: We compare trajectories computed after integration for relevant cell types that depict a continuous cellular differentiation process to trajectories computed per batch and modality using this metric.
Revised: We compare trajectories computed after integration for relevant cell types that depict a continuous cellular differentiation process to trajectories computed per batch and modality using this metric.

Original: The conservation of the trajectory is quantified via Spearman’s rank correlation coefficient [23], s , between the pseudotime values before and after integration.
Revised: The conservation of the trajectory is quantified via Spearman’s rank correlation coefficient [23], s , between the pseudotime values before and after integration.

Original: The final score is scaled to a value between 0 and 1 using the equation:
Revised: The final score is scaled to a value between 0 and 1 using the equation:

Original: [Equation]
Revised: [Equation]

Original: where a value of 1 or 0 indicates that the cells on the trajectory are in the same order before and after integration, or in the reverse order.
Revised: where a value of 1 or 0 indicates that the cells on the trajectory are in the same order before and after integration, or in the reverse order.

Original: A.2 Batch Removal Metrics
Revised: A.2 Batch Removal Metrics

Original: Batch ASW:
Revised: Batch ASW:

Original: The ASW
Revised: The ASW

Original: is
Revised: is

Original: used
Revised: used

Original: to
Revised: to

Original: quantify batch mixing
Revised: quantify batch mixing

Original: by taking into account
Revised: by taking into account

Original: the
Revised: the

Original: incompatibility of batch
Revised: incompatibility of batch

Original: labels
Revised: labels

Original: per cell type cluster.
Revised: per cell type cluster.

Original: We consider the absolute silhouette
Revised: We consider the absolute silhouette

Original: width
Revised: width

Original: on
Revised: on

Original: batch labels per
Revised: batch labels per

Original: cell,
Revised: cell,

Original: in particular.
Revised: in particular.

Original: Here,
Revised: Here,

Original: zero
Revised: zero

Original: shows that batches are thoroughly mixed, but any variation
Revised: shows that batches are thoroughly mixed, but any variation

Original: from
Revised: from

Original: zero indicates the presence of a batch effect.
Revised: zero indicates the presence of a batch effect.

Original: We rescale this score so that higher values imply better batch mixing and use the equation below to determine the per-cell type label, j:
Revised: We rescale this score so that higher values imply better batch mixing and use the equation below to determine the per-cell type label, j:

Original: [Equation]
Revised: [Equation]

Original: where C j is the set of cells with the cell label j and | C j | denotes the number of cells in that set.
Revised: where C j is the set of cells with the cell label j and | C j | denotes the number of cells in that set.

Original: To obtain the final batchASW score, the label-specific batchASW j scores are averaged:
Revised: To obtain the final batchASW score, the label-specific batchASW j scores are averaged:

Original: [Equation]
Revised: [Equation]

Original: where M is the set of unique cell labels.
Revised: where M is the set of unique cell labels.

Original: A batchASW value of 1 suggests optimal batch mixing, whereas a value of 0 indicates severely separated batches.
Revised: A batchASW value of 1 suggests optimal batch mixing, whereas a value of 0 indicates severely separated batches.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: Graph connectivity: The graph connectivity metric determines whether cells of the same kind from various batches are embedded close to one another.
Revised: Graph connectivity: The graph connectivity metric determines whether cells of the same kind from various batches are embedded close to one another.

Original: This is determined by computing a k-nearest neighbor (kNN) graph using Euclidean distances on the embedding.
Revised: This is determined by computing a k-nearest neighbor (kNN) graph using Euclidean distances on the embedding.

Original: Then, we determine if all cells with the same cell identity label are connected in this kNN graph.
Revised: Then, we determine if all cells with the same cell identity label are connected in this kNN graph.

Original: For each cell identity label c , we generate the subset kNN graph G =
Revised: For each cell identity label c , we generate the subset kNN graph G =

Original: ( N c ; E c ) , which contains only cells from a given label.
Revised: ( N c ; E c ) , which contains only cells from a given label.

Original: Using these subset kNN graphs, we compute the graph connectivity score:
Revised: Using these subset kNN graphs, we compute the graph connectivity score:

Original: [Equation]
Revised: [Equation]

Original: (13) where C represents the set of cell identity labels, LCC () denotes the number of nodes in the largest connected component of the graph, and N c is the number of nodes with cell identity c .
Revised: (13) where C represents the set of cell identity labels, LCC () denotes the number of nodes in the largest connected component of the graph, and N c is the number of nodes with cell identity c .

Original: The resulting score ranges from 0 to 1, where 1 means that all cells with the same cell identity are connected in the integrated kNN graph, while 0 indicates that no cell is connected in the network.
Revised: The resulting score ranges from 0 to 1, where 1 means that all cells with the same cell identity are connected in the integrated kNN graph, while 0 indicates that no cell is connected in the network.

Original: A.3 Metric Aggregation
Revised: A.3 Metric Aggregation

Original: Due to the differing nature of each metric, each metric would be assigned with a weight.
Revised: Due to the differing nature of each metric, each metric would be assigned with a weight.

Original: An overall weighted average of batch correction and bio-conservation scores will be computed via the equation:
Revised: An overall weighted average of batch correction and bio-conservation scores will be computed via the equation:

Original: [Equation]
Revised: [Equation]

Original: B Data Statistics
Revised: B Data Statistics

Original: The Table 4 and Table 5 provide statistics about dataset used in modality prediction task and modality matching task respectively.
Revised: The Table 4 and Table 5 provide statistics about dataset used in modality prediction task and modality matching task respectively.

Original: C Experimental Setup
Revised: C Experimental Setup

Original: Note that in this experiment, we follow the official settings and datasets in the multimodal single-cell data integration competition at NeurIPS 2021 [10] and we compare the performance of the proposed framework with that from the top winners in the competition.
Revised: Note that in this experiment, we follow the official settings and datasets in the multimodal single-cell data integration competition at NeurIPS 2021 [10] and we compare the performance of the proposed framework with that from the top winners in the competition.

Original: C.1 Modality Prediction
Revised: C.1 Modality Prediction

Original: Datasets.
Revised: Datasets.

Original: In the modality prediction dataset, each modality is provided with source data, preprocessed data (with default methods), and batch labels.
Revised: In the modality prediction dataset, each modality is provided with source data, preprocessed data (with default methods), and batch labels.

Original: Data statistics are shown in Table 4 in Appendix B.
Revised: Data statistics are shown in Table 4 in Appendix B.

Original: Note that the GEX-ATAC and ATAC-GEX subtasks are not entirely symmetric, because in the
Revised: Note that the GEX-ATAC and ATAC-GEX subtasks are not entirely symmetric, because in the

Original: GEX-ATAC task, the output dimension is deliberately reduced, where 10,000 ATAC features are randomly selected out of the original 116,490 dimensions.
Revised: GEX-ATAC task, the output dimension is deliberately reduced, where 10,000 ATAC features are randomly selected out of the original 116,490 dimensions.

Original: Settings and Parameters.
Revised: Settings and Parameters.

Original: For our own model, we report the average performance of 10 runs.
Revised: For our own model, we report the average performance of 10 runs.

Original: In each run, we reserved 15% of training cells for validation and early stopping.
Revised: In each run, we reserved 15% of training cells for validation and early stopping.

Original: In practice, several tricks help boost the performance: (1) utilizing initial residual instead of the skip connections in Eq.5, (2) using group normalization for aggregation results and dismissing the edge weight normalization stated.
Revised: In practice, several tricks help boost the performance: (1) utilizing initial residual instead of the skip connections in Eq.5, (2) using group normalization for aggregation results and dismissing the edge weight normalization stated.

Original: Besides, we empirically set our parameter α to 0 .
Revised: Besides, we empirically set our parameter α to 0 .

Original: Additionally, in order to fit the high-dimensional
Revised: Additionally, in order to fit the high-dimensional

Original: ATAC features, we enabled node sampling.
Revised: ATAC features, we enabled node sampling.

Original: Baselines.
Revised: Baselines.

Original: In Table 1, we only show the teams that acquired top results in one or more subtasks in the competition, because they are officially required to declare their model details.
Revised: In Table 1, we only show the teams that acquired top results in one or more subtasks in the competition, because they are officially required to declare their model details.

Original: For further comparison, we briefly detail each method: (1) Baseline, a truncated-SVD dimensionality reduction followed by linear regression.
Revised: For further comparison, we briefly detail each method: (1) Baseline, a truncated-SVD dimensionality reduction followed by linear regression.

Original: (2) Dengkw, a well-designed model based on kernel ridge regression.
Revised: (2) Dengkw, a well-designed model based on kernel ridge regression.

Original: (3) Novel, an encoder-decoder structure with LSI preprocessing.
Revised: (3) Novel, an encoder-decoder structure with LSI preprocessing.

Original: (4) Living System Lab, an ensemble model composed of random forest models, catboost[22] models, and k-nearest neighbors regression models.
Revised: (4) Living System Lab, an ensemble model composed of random forest models, catboost[22] models, and k-nearest neighbors regression models.

Original: (5) Cajal, a feed forward neural network with heavy feature selection guided by prior knowledge.
Revised: (5) Cajal, a feed forward neural network with heavy feature selection guided by prior knowledge.

Original: ScJoint, an ensemble neural network model incorporated various strategies of preprocessing such as extracting TF-IDF features and filtering highly variable genes/features.
Revised: ScJoint, an ensemble neural network model incorporated various strategies of preprocessing such as extracting TF-IDF features and filtering highly variable genes/features.

Original: C.2 Modality Matching
Revised: C.2 Modality Matching

Original: Datasets.
Revised: Datasets.

Original: The majority of the modality matching dataset is the same as the modality prediction dataset (as shown in Table 5 in Appendix B, while several differences exist, including: (1) the number of testing cells; (2) the dimensionality of ATAC features; and (3) the inconsistent cell order among modalities.
Revised: The majority of the modality matching dataset is the same as the modality prediction dataset (as shown in Table 5 in Appendix B, while several differences exist, including: (1) the number of testing cells; (2) the dimensionality of ATAC features; and (3) the inconsistent cell order among modalities.

Original: In the training data, samples’ correspondence between different modalities is given, while for the test data, our goal is to find the correspondence.
Revised: In the training data, samples’ correspondence between different modalities is given, while for the test data, our goal is to find the correspondence.

Original: Settings and Parameters.
Revised: Settings and Parameters.

Original: The experimental settings are similar to the previous task, while some adjustments were made.
Revised: The experimental settings are similar to the previous task, while some adjustments were made.

Original: For calculation convenience, we decoupled the propagation and transformation.
Revised: For calculation convenience, we decoupled the propagation and transformation.

Original: Besides, batch labels are given in company with the test data, which provides a very strong prior knowledge for matching.
Revised: Besides, batch labels are given in company with the test data, which provides a very strong prior knowledge for matching.

Original: We thus divided the test data into different batches, then matched samples that belong to the same batch.
Revised: We thus divided the test data into different batches, then matched samples that belong to the same batch.

Original: This trick dramatically reduced the search space, resulting in a significant performance boost.
Revised: This trick dramatically reduced the search space, resulting in a significant performance boost.

Original: To be fair, we have confirmed that the winning solution also used the same strategy.
Revised: To be fair, we have confirmed that the winning solution also used the same strategy.

Original: Baselines.
Revised: Baselines.

Original: In Table 2, we only compare our models with the winning team and runner-up team in the competition.
Revised: In Table 2, we only compare our models with the winning team and runner-up team in the competition.

Original: Next we briefly introduce those models: (1) Baseline, first projecting one modality to the other with linear regression, and then searching for the nearest neighbors in the same modality.
Revised: Next we briefly introduce those models: (1) Baseline, first projecting one modality to the other with linear regression, and then searching for the nearest neighbors in the same modality.

Original: (2) GLUE [5] (CLUE), the official winner, a variational auto-encoder (VAE) model supervised by three auxiliary losses.
Revised: (2) GLUE [5] (CLUE), the official winner, a variational auto-encoder (VAE) model supervised by three auxiliary losses.

Original: (3) Novel, a feed-forward neural network directly supervised by matching loss.
Revised: (3) Novel, a feed-forward neural network directly supervised by matching loss.

Original: C.3 Joint Embedding
Revised: C.3 Joint Embedding

Original: Datasets.
Revised: Datasets.

Original: The training data of this task are basically the same as the modality prediction task.
Revised: The training data of this task are basically the same as the modality prediction task.

Original: Moreover, data from different modalities are already aligned.
Revised: Moreover, data from different modalities are already aligned.

Original: Regarding the complementary data, there are two settings for this task.
Revised: Regarding the complementary data, there are two settings for this task.

Original: In the ’online’ setting, the only data is features of two modalities.
Revised: In the ’online’ setting, the only data is features of two modalities.

Original: Meanwhile, in the ’pre-trained’ setting, any external knowledge is acceptable.
Revised: Meanwhile, in the ’pre-trained’ setting, any external knowledge is acceptable.

Original: In this paper, we follow the second setting (i.e. pre-trained setting) and we only compare our results with other pre-trained models.
Revised: In this paper, we follow the second setting (i.e. pre-trained setting) and we only compare our results with other pre-trained models.

Original: Generally speaking, pre-trained models obtain better performance than online models.
Revised: Generally speaking, pre-trained models obtain better performance than online models.

Original: In our experiments, cell type labels are provided in the training phase, while test data consist of all the train cells and unseen test cells but are not along with cell type labels.
Revised: In our experiments, cell type labels are provided in the training phase, while test data consist of all the train cells and unseen test cells but are not along with cell type labels.

Original: Settings and Parameters.
Revised: Settings and Parameters.

Original: Considering that the input in this task is similar to modality matching, we followed settings in Section C.2.
Revised: Considering that the input in this task is similar to modality matching, we followed settings in Section C.2.

Original: Baselines.
Revised: Baselines.

Original: We briefly describe the other three models in Table 3: (1) Baseline, a concatenation of PCA results of two modalities.
Revised: We briefly describe the other three models in Table 3: (1) Baseline, a concatenation of PCA results of two modalities.

Original: (2) Amateur (JAE), the official winner, an auto-encoder model that incorporates extra supervision (cell annotations).
Revised: (2) Amateur (JAE), the official winner, an auto-encoder model that incorporates extra supervision (cell annotations).

Original: The model is adapted from scDEC [
Revised: The model is adapted from scDEC [