Original: S INGLE -C ELL
Revised: S INGLE -C ELL

Original: C APSULE
Revised: C APSULE

Original: A TTENTION :
Revised: A TTENTION :

Original: AN
Revised: AN

Original: INTER -
Revised: INTER -

Original: PRETABLE METHOD OF CELL TYPE CLASSIFICATION FOR SINGLE - CELL RNA- SEQUENCING DATA
Revised: PRETABLE METHOD OF CELL TYPE CLASSIFICATION FOR SINGLE - CELL RNA- SEQUENCING DATA

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Single-cell RNA-sequencing technique can obtain genes’ expression level of every cell.
Revised: Single-cell RNA-sequencing technique can obtain genes’ expression level of every cell.

Original: Cell type classiﬁcation (also known as cell type annotation) on singlecell RNA-seq data helps to explore cellular heterogeneity and diversity.
Revised: Cell type classiﬁcation (also known as cell type annotation) on singlecell RNA-seq data helps to explore cellular heterogeneity and diversity.

Original: Previous methods for cell type classiﬁcation are either based on statistical hypotheses of gene expression or deep neural networks.
Revised: Previous methods for cell type classiﬁcation are either based on statistical hypotheses of gene expression or deep neural networks.

Original: However, the hypotheses may not reﬂect the true expression level.
Revised: However, the hypotheses may not reﬂect the true expression level.

Original: Deep neural networks lack interpretation for the result.
Revised: Deep neural networks lack interpretation for the result.

Original: Here we present an interpretable neural-network based method single-cell capsule attention(scCA) which assigns cells to different cell types based on their different feature patterns.
Revised: Here we present an interpretable neural-network based method single-cell capsule attention(scCA) which assigns cells to different cell types based on their different feature patterns.

Original: In our model, we ﬁrst generate capsules which extract different features of the cells.
Revised: In our model, we ﬁrst generate capsules which extract different features of the cells.

Original: Then we obtain compound features which combine a set of features’ information through a LSTM model.
Revised: Then we obtain compound features which combine a set of features’ information through a LSTM model.

Original: In the end, we train attention weights and apply them to the compound features.
Revised: In the end, we train attention weights and apply them to the compound features.

Original: scCA provides a strong interpretation for cell type classiﬁcation result.
Revised: scCA provides a strong interpretation for cell type classiﬁcation result.

Original: Cells from the same cell type share a similar pattern of capsules’ relationship and similar distribution of attention weights for compound features.
Revised: Cells from the same cell type share a similar pattern of capsules’ relationship and similar distribution of attention weights for compound features.

Original: Compared with previous methods for cell type classiﬁcation on nine datasets, scCA shows high accuracy on all datasets with robustness and reliable interpretation.
Revised: Compared with previous methods for cell type classiﬁcation on nine datasets, scCA shows high accuracy on all datasets with robustness and reliable interpretation.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Genes’ expression on celluar level provides lots of information for us to explore homogeneity and heterogeneity among different cells(Liang et al., 2014; Muraro et al., 2016; Baron et al., 2016).
Revised: Genes’ expression on celluar level provides lots of information for us to explore homogeneity and heterogeneity among different cells(Liang et al., 2014; Muraro et al., 2016; Baron et al., 2016).

Original: Traditional bulk sequencing technique can only measure the average gene expression level of all cells in a sample.
Revised: Traditional bulk sequencing technique can only measure the average gene expression level of all cells in a sample.

Original: Compared with bulk data, single-cell RNA-sequencing(single-cell RNA-seq) technique gives a more accurate measurement of every cell.
Revised: Compared with bulk data, single-cell RNA-sequencing(single-cell RNA-seq) technique gives a more accurate measurement of every cell.

Original: Though single-cell RNA-sequencing is a rising technique, it is not mature enough and still has some limitations.
Revised: Though single-cell RNA-sequencing is a rising technique, it is not mature enough and still has some limitations.

Original: For example, single-cell
Revised: For example, single-cell

Original: RNA-seq data has high ’dropout’.
Revised: RNA-seq data has high ’dropout’.

Original: ’Dropout’ means zero or low read counts in the data because many genes’ expression are hard to detect(Huang et al., 2018; Pierson & Yau, 2015).
Revised: ’Dropout’ means zero or low read counts in the data because many genes’ expression are hard to detect(Huang et al., 2018; Pierson & Yau, 2015).

Original: Also, singlecell RNA-seq data are of high dimension because thousands of genes are in the transcriptome.
Revised: Also, singlecell RNA-seq data are of high dimension because thousands of genes are in the transcriptome.

Original: These all bring challenges for studies on single-cell RNA-seq data.
Revised: These all bring challenges for studies on single-cell RNA-seq data.

Original: So far, many data mining and analysis methods have been applied on single-cell RNA-seq data to solve these challenges and explore more information on cells and genes(Svensson et al., 2018; Han et al., 2018; Lopez et al., 2018; Hwang et al., 2018; Guo et al., 2015).
Revised: So far, many data mining and analysis methods have been applied on single-cell RNA-seq data to solve these challenges and explore more information on cells and genes(Svensson et al., 2018; Han et al., 2018; Lopez et al., 2018; Hwang et al., 2018; Guo et al., 2015).

Original: Cell type classiﬁcation is one of the most important tasks in single-cell RNA-seq data analysis.
Revised: Cell type classiﬁcation is one of the most important tasks in single-cell RNA-seq data analysis.

Original: It helps to identify different cell types and explore the cellular heterogeneity.
Revised: It helps to identify different cell types and explore the cellular heterogeneity.

Original: Previous methods on cell type classiﬁcation can be categorized into methods based on statistical distribution hypotheses of gene expression and deep learning methods based on neural networks(Abdelaal et al., 2019).
Revised: Previous methods on cell type classiﬁcation can be categorized into methods based on statistical distribution hypotheses of gene expression and deep learning methods based on neural networks(Abdelaal et al., 2019).

Original: These methods lay a solid foundation for cell type classiﬁcation.
Revised: These methods lay a solid foundation for cell type classiﬁcation.

Original: However, they still have limitations and weaknesses.
Revised: However, they still have limitations and weaknesses.

Original: Methods based on statistical hypotheses rely on prior knowledge of marker genes or
Revised: Methods based on statistical hypotheses rely on prior knowledge of marker genes or

Original: gene expression’s distribution assumptions.
Revised: gene expression’s distribution assumptions.

Original: Up to now, there is no widely recognized hypotheses of gene expression.
Revised: Up to now, there is no widely recognized hypotheses of gene expression.

Original: Deep neural network(DNN), a fast and effective learning method, is widely used in various ﬁelds.
Revised: Deep neural network(DNN), a fast and effective learning method, is widely used in various ﬁelds.

Original: DNN does not rely on distribution hypotheses, however, it’s hard to provide an reasonable explanation for the result behind the deep ’black-box’(Lin et al., 2017).
Revised: DNN does not rely on distribution hypotheses, however, it’s hard to provide an reasonable explanation for the result behind the deep ’black-box’(Lin et al., 2017).

Original: ACTINN(Ma &
Revised: ACTINN(Ma &

Original: Pellegrini, 2020), a deep neural network based method for cell type classiﬁcation, uses three hidden layers in its architecture.
Revised: Pellegrini, 2020), a deep neural network based method for cell type classiﬁcation, uses three hidden layers in its architecture.

Original: Though its model can predict cell types, the model still lacks reliable interpretations for the result.
Revised: Though its model can predict cell types, the model still lacks reliable interpretations for the result.

Original: The challenge is that DNN does not provide a recognizable pattern that is closely related to different cell types.
Revised: The challenge is that DNN does not provide a recognizable pattern that is closely related to different cell types.

Original: Here, we introduce a new neural network based method scCA for cell type classiﬁcation.
Revised: Here, we introduce a new neural network based method scCA for cell type classiﬁcation.

Original: scCA provides an reasonable interpretation for its classiﬁcation result.
Revised: scCA provides an reasonable interpretation for its classiﬁcation result.

Original: Our model learns features from cells and provides patterns behind these features that are related to cell types.
Revised: Our model learns features from cells and provides patterns behind these features that are related to cell types.

Original: The ﬁrst part is feature extraction.
Revised: The ﬁrst part is feature extraction.

Original: Given the gene expression level in every cell, We use capsules to extract different features.
Revised: Given the gene expression level in every cell, We use capsules to extract different features.

Original: A capsule is a vector that represents a feature, which captures a certain type of information of the cell.
Revised: A capsule is a vector that represents a feature, which captures a certain type of information of the cell.

Original: Capsules also serve as dimension reduction for single cell RNA-seq data.
Revised: Capsules also serve as dimension reduction for single cell RNA-seq data.

Original: We calculate the Pearson coefﬁcients between every two capsules and generate the coefﬁcient heatmap of every cell.
Revised: We calculate the Pearson coefﬁcients between every two capsules and generate the coefﬁcient heatmap of every cell.

Original: We discover that cells from same cell type share a similar pattern in the heatmap while cells from different cell types have different patterns.
Revised: We discover that cells from same cell type share a similar pattern in the heatmap while cells from different cell types have different patterns.

Original: These capsules capture features that are important for cell type classiﬁcation.
Revised: These capsules capture features that are important for cell type classiﬁcation.

Original: And then, we use a sequence to sequence model, bidirectional-LSTM, to generate compound features.
Revised: And then, we use a sequence to sequence model, bidirectional-LSTM, to generate compound features.

Original: Each compound feature combines a set of features’ information.
Revised: Each compound feature combines a set of features’ information.

Original: Furthermore, we train multiple attention weights for these compound features.
Revised: Furthermore, we train multiple attention weights for these compound features.

Original: Then we multiply these weights to the compound features and generate the classiﬁcation result.
Revised: Then we multiply these weights to the compound features and generate the classiﬁcation result.

Original: Attention weights help to lay more emphasis on important compound features.
Revised: Attention weights help to lay more emphasis on important compound features.

Original: The larger the attention weight is, the more crucial its corresponding compound feature is.
Revised: The larger the attention weight is, the more crucial its corresponding compound feature is.

Original: we also draw heatmaps for every cells’ attention weights.
Revised: we also draw heatmaps for every cells’ attention weights.

Original: We discover that cells from same cell type have similar patterns in the heatmaps of attention weights.
Revised: We discover that cells from same cell type have similar patterns in the heatmaps of attention weights.

Original: This leads to a better classiﬁcation performance and also provides an reliable interpretation behind the network.
Revised: This leads to a better classiﬁcation performance and also provides an reliable interpretation behind the network.

Original: We evaluate scCA on several datasets and compare with previous work.
Revised: We evaluate scCA on several datasets and compare with previous work.

Original: Our method shows high accuracy and robustness on all datasets.
Revised: Our method shows high accuracy and robustness on all datasets.

Original: The main contributions of our article are as follow: We propose an interpretable neural network based method scCA for cell type classiﬁcation.
Revised: The main contributions of our article are as follow: We propose an interpretable neural network based method scCA for cell type classiﬁcation.

Original: Through heatmaps of capsules’ Pearson coefﬁcients and attention weights, we ﬁnd that cells from same cell type share a similar pattern.
Revised: Through heatmaps of capsules’ Pearson coefﬁcients and attention weights, we ﬁnd that cells from same cell type share a similar pattern.

Original: scCA achieves high accuracy and stable performances on all datasets.
Revised: scCA achieves high accuracy and stable performances on all datasets.

Original:  R ELATED W ORK Statistical Hypotheses Based Model
Revised:  R ELATED W ORK Statistical Hypotheses Based Model

Original: Many machine learning methods with statistical hypotheses of gene expression level have been proposed for cell type classiﬁcation.
Revised: Many machine learning methods with statistical hypotheses of gene expression level have been proposed for cell type classiﬁcation.

Original: Some of these methods suggest distribution hypotheses of gene expression level and employ machine learning models based on these hypotheses.
Revised: Some of these methods suggest distribution hypotheses of gene expression level and employ machine learning models based on these hypotheses.

Original: Some build a hierarchy with prior knowledge and statistical hypotheses.
Revised: Some build a hierarchy with prior knowledge and statistical hypotheses.

Original: Moana takes advantage of KNN and
Revised: Moana takes advantage of KNN and

Original: SVM model(Wagner & Yanai, 2018).
Revised: SVM model(Wagner & Yanai, 2018).

Original: It ﬁrst preprocesses data with KNN, then it uses dimension reduction method to extract useful features, at last it uses a Support Vector Machine classiﬁer.
Revised: It ﬁrst preprocesses data with KNN, then it uses dimension reduction method to extract useful features, at last it uses a Support Vector Machine classiﬁer.

Original: In addition, Moana uses marker gene as a prior knowledge for cell type identiﬁcation.
Revised: In addition, Moana uses marker gene as a prior knowledge for cell type identiﬁcation.

Original: Different from
Revised: Different from

Original: Moana, Garnett proposes a model based on tree hierarchy.
Revised: Moana, Garnett proposes a model based on tree hierarchy.

Original: It discovers the relation between cell types and subtypes with a tree structure(Pliner et al., 2019).
Revised: It discovers the relation between cell types and subtypes with a tree structure(Pliner et al., 2019).

Original: With the help of marker gene, it builds a linear model to classify cell types.
Revised: With the help of marker gene, it builds a linear model to classify cell types.

Original: CellAssign builds a probabilistic model based on prior knowledge to classify cell types(Zhang et al., 2019b).
Revised: CellAssign builds a probabilistic model based on prior knowledge to classify cell types(Zhang et al., 2019b).

Original: SCINA makes use of EM algorithm and the information of marker gene to accomplish clustering as well as cell type identiﬁcation task(Zhang et al., 2019a).
Revised: SCINA makes use of EM algorithm and the information of marker gene to accomplish clustering as well as cell type identiﬁcation task(Zhang et al., 2019a).

Original: scPred extracts features using SVD, then with a nonlinear kernel SVM, it predicts different cell
Revised: scPred extracts features using SVD, then with a nonlinear kernel SVM, it predicts different cell

Original: types(Alquicira-Hernandez et al., 2019).
Revised: types(Alquicira-Hernandez et al., 2019).

Original: However, these methods have their limitations.
Revised: However, these methods have their limitations.

Original: Their hypotheses  of gene expression may notmatch with the true expression level.
Revised: Their hypotheses such as ZINB(Zero-inﬂated Negative Binomial) or NB distribution of gene expression are not the exact reﬂections of gene ’s expression levels in cells. Actually, there is not a consensus statistical distribution for single-cell RNA-seq data.

Original: Also they need prior knowledge like marker gene for cell type classiﬁcation.
Revised: Also they need prior knowledge like marker gene for cell type classiﬁcation.

Original: Neural Network Based Models
Revised: Neural Network Based Models

Original: Deep neural network(DNN) based methods have been applied to cell type classiﬁcation task.
Revised: Deep neural network(DNN) based methods have been applied to cell type classiﬁcation task.

Original: They are fast and robust.
Revised: They are fast and robust.

Original: Neural networks don’t heavily rely on prior knowledge and statistical hypotheses.
Revised: Neural networks don’t heavily rely on prior knowledge and statistical hypotheses.

Original: ACTINN uses three fully connected layers and ReLU, softmax activation function to classify different cell types(Ma & Pellegrini, 2020).
Revised: ACTINN uses three fully connected layers and ReLU, softmax activation function to classify different cell types(Ma & Pellegrini, 2020).

Original: scCapsNet builds a capsule network to extract features from the data and use dynamic routings among capsules to get a classiﬁcation result(Wang et al., 
Revised: scCapsNet builds a capsule network to extract features from the data and use dynamic routings among capsules to get a classiﬁcation result(Wang et al., 2020).

Original: However, neural networks lack interpretation for their result.
Revised: However, neural networks lack interpretation for their result.

Original: Through training the whole network by propagation, it’s hard to explain every component’s contribution for the result.
Revised: Through training the whole network by propagation, it’s hard to explain every component’s contribution for the result.

Original:  P RELIMINARIES Capsule Network and Dynamic Routing
Revised:  P RELIMINARIES Capsule Network and Dynamic Routing

Original: Hinton ﬁrst proposed Capsule network and dynamic routing in computer vision(Sabour et al., 2017).
Revised: Hinton, Sabour and other co-authors ﬁrst proposed Capsule network and dynamic routing in computer vision(Sabour et al., 2017).

Original: A capsule is a set of neurons or a vector that can represent an entity.
Revised: A capsule is a set of neurons or a vector that can represent an entity.

Original: Dynamic routing is the mechanism between capsule’s layers.
Revised: Dynamic routing is the mechanism between capsule’s layers.

Original: As Hinton points out, the capsules in the layer above are parents and the capsules in the layer below are children.(Hinton et al., 2000; 2011).
Revised: As they point out, the capsules in the layer above are parents and the capsules in the layer below are children.(Hinton et al., 2000; 2011).

Original: Dynamic routing helps to send the results from capsules in children layer to its appropriate parents in the above layer.
Revised: Dynamic routing helps to send the results from capsules in children layer to its appropriate parents in the above layer.

Original: By adjusting the coupling coefﬁcients between capsules in two layers, the network learns which parent capsule is more important for the children capsule.
Revised: By adjusting the coupling coefﬁcients between capsules in two layers, the network learns which parent capsule is more important for the children capsule.

Original: Here in scCA, we use capsules to capture different features of the cell.
Revised: Here in scCA, we use capsules to capture different features of the cell.

Original: We use a vector for implementation of capsule in our model.
Revised: We use a vector for implementation of capsule in our model.

Original: However, different from dynamic routing, we use LSTM and attention as the architecture for cell type classiﬁcation.
Revised: However, different from dynamic routing, we use LSTM and attention as the architecture for cell type classiﬁcation.

Original: Long Short-Term Memory and Attention
Revised: Long Short-Term Memory and Attention

Original: Long Short-Term Memory(LSTM) is a model widely used in Natural Language Processing(NLP)(Hochreiter & Schmidhuber, 1997).
Revised: Long Short-Term Memory(LSTM) is a model widely used in Natural Language Processing(NLP)(Hochreiter & Schmidhuber, 1997).

Original: It is a sequence to sequence model.
Revised: It is a sequence to sequence model.

Original: The inputs are usually the word embeddings(representations) of a sentence and the output is a sequence of representations.
Revised: The inputs are usually the word embeddings(representations) of a sentence and the output is a sequence of representations.

Original: Each output representation not only contains information from its corresponding input, but also information from representations in the context, especially its neighbor inputs.
Revised: Each output representation not only contains information from its corresponding input, but also information from representations in the context, especially its neighbor inputs.

Original: In our method, after extracting different features into capsules, we input these capsules into the
Revised: In our method, after extracting different features into capsules, we input these capsules into the

Original: LSTM model and obtain the output sequence.
Revised: LSTM model and obtain the output sequence.

Original: Every vector in the output sequence contains information of its corresponding input feature as well as other features.
Revised: Every vector in the output sequence contains information of its corresponding input feature as well as other features.

Original: We name these vectors compound features .
Revised: We name these vectors compound features .

Original: Some of the compound features have closer relationships with certain cell types.
Revised: Some of the compound features have closer relationships with certain cell types.

Original: Inspired by Attention(Vaswani et al., 2017) mechanism, we apply attention weights to compound features in order to differentiate and emphasize those important compound features for classiﬁcation.
Revised: Inspired by Attention(Vaswani et al., 2017) mechanism, we apply attention weights to compound features in order to differentiate and emphasize those important compound features for classiﬁcation.

Original:  SINGLE - CELL C APSULE A TTENTION
Revised:  SINGLE - CELL C APSULE A TTENTION

Original: In this section, we provide a detailed description of single-cell Capsule Attention(scCA).
Revised: In this section, we provide a detailed description of single-cell Capsule Attention(scCA).

Original: We will also give a vivid and transparent explanation for our classiﬁcation result.
Revised: We will also give a vivid and transparent explanation for our classiﬁcation result.

Original: scCA can be divided into three parts.
Revised: scCA can be divided into three parts.

Original: The ﬁgure below shows its architecture.
Revised: The ﬁgure below shows its architecture.

Original: 4. F EATURE EXTRACTION
Revised: 4. F EATURE EXTRACTION

Original: The ﬁrst part of scCA is feature extraction.
Revised: The ﬁrst part of scCA is feature extraction.

Original: Given thousands of genes’ expression levels for a cell, we hope to learn simple but efﬁcacious features for classiﬁcation.
Revised: Given thousands of genes’ expression levels for a cell, we hope to learn simple but efﬁcacious features for classiﬁcation.

Original: Capsules not only serve as dimension reduction, but also extract a type of information of the cell.
Revised: Capsules not only serve as dimension reduction, but also extract a type of information of the cell.

Original: Through fully connected layers, we generate capsules to extract different features.
Revised: Through fully connected layers, we generate capsules to extract different features.

Original: [Equation]
Revised: [Equation]

Original: In this formula, m represents the number of capsules.
Revised: In this formula, m represents the number of capsules.

Original: W and b are the weight matrices of fully connected layers.
Revised: W and b are the weight matrices of fully connected layers.

Original: ReLU is the activation function we use.
Revised: ReLU is the activation function we use.

Original: Every capsule contains a feature of a cell.
Revised: Every capsule contains a feature of a cell.

Original: We hope to discover different patterns of these features for different cell types.
Revised: We hope to discover different patterns of these features for different cell types.

Original: We calculate the Pearson coefﬁcients between every two capsules.
Revised: We calculate the Pearson coefﬁcients between every two capsules.

Original: The
Revised: The

Original: Pearson coefﬁcient is an indication of relationship between two features behind capsules.
Revised: Pearson coefﬁcient is an indication of relationship between two features behind capsules.

Original: For every cell, we come to a matrix where every element is a Pearson coefﬁcient of two capsules.
Revised: For every cell, we come to a matrix where every element is a Pearson coefﬁcient of two capsules.

Original: We then draw a heatmap of the matrix.
Revised: We then draw a heatmap of the matrix.

Original: We discover that cells from same cell type share a similar pattern while cells from different cell types differ from each other.
Revised: We discover that cells from same cell type share a similar pattern while cells from different cell types differ from each other.

Original: Taking six cells from dataset GSE72056 as examples, we can see cells from same cell type have a similar pattern of capsules’ relationship.
Revised: Taking six cells from dataset GSE72056 as examples, we can see cells from same cell type have a similar pattern of capsules’ relationship.

Original: The deeper the color is in the heatmap, the closer the relationship of two capsules is.
Revised: The deeper the color is in the heatmap, the closer the relationship of two capsules is.

Original: The pattern is distinguishable among different cell types.
Revised: The pattern is distinguishable among different cell types.

Original: From the ﬁgures above, we conclude that there exists some particular feature patterns for different cell types.
Revised: From the ﬁgures above, we conclude that there exists some particular feature patterns for different cell types.

Original: Our feature extraction really extracts useful and discriminative features.
Revised: Our feature extraction really extracts useful and discriminative features.

Original: 4. LSTM F OR C OMPOUND F EATURES
Revised: 4. LSTM F OR C OMPOUND F EATURES

Original: Single feature is not enough for cell type classiﬁcation because cells from different cell types may share same features.
Revised: Single feature is not enough for cell type classiﬁcation because cells from different cell types may share same features.

Original: It’s the speciﬁc combination of features that decides the cell type.
Revised: It’s the speciﬁc combination of features that decides the cell type.

Original: Moreover, some features may have special relationships with other features.
Revised: Moreover, some features may have special relationships with other features.

Original: We need to explore the special patterns behind the features that determine cell types.
Revised: We need to explore the special patterns behind the features that determine cell types.

Original: The motivation for compound features lays in here.
Revised: The motivation for compound features lays in here.

Original: A LSTM model captures other features’ information in the context.
Revised: A LSTM model captures other features’ information in the context.

Original: The capsules generated in feature extraction serve as the input of LSTM.
Revised: The capsules generated in feature extraction serve as the input of LSTM.

Original: For every feature in the sequence, a LSTM model takes in hidden information from other features as well as its own information.
Revised: For every feature in the sequence, a LSTM model takes in hidden information from other features as well as its own information.

Original: Then LSTM model outputs a sequence of compound features.
Revised: Then LSTM model outputs a sequence of compound features.

Original: For implementation, we use a bi-directional LSTM model for training which is more stable and efﬁcient.
Revised: For implementation, we use a bi-directional LSTM model for training which is more stable and efﬁcient.

Original: [Equation]
Revised: [Equation]

Original: 4. A TTENTION M ECHANISM
Revised: 4. A TTENTION M ECHANISM

Original: In the last part of scCA, scCA learns attention weights for these compound features.
Revised: In the last part of scCA, scCA learns attention weights for these compound features.

Original: We have two intentions for these attention weights.
Revised: We have two intentions for these attention weights.

Original: The ﬁrst intention is that we want to eliminate the effect of capsule’s order in the input sequence.
Revised: The ﬁrst intention is that we want to eliminate the effect of capsule’s order in the input sequence.

Original: LSTM model is widely used in natural language processing where input data is a sequence.
Revised: LSTM model is widely used in natural language processing where input data is a sequence.

Original: However, for cell type classiﬁcation, capsules or features’ order in the sequence shouldn’t inﬂuence the result.
Revised: However, for cell type classiﬁcation, capsules or features’ order in the sequence shouldn’t inﬂuence the result.

Original: So we add attention to emphasize the compound features’ information itself, rather than its position information in the sequence.
Revised: So we add attention to emphasize the compound features’ information itself, rather than its position information in the sequence.

Original: The second intention is that some compound features may be more signiﬁcant in cell type classiﬁcation.
Revised: The second intention is that some compound features may be more signiﬁcant in cell type classiﬁcation.

Original: The attention weights for these compound features will be large, showing their importance.
Revised: The attention weights for these compound features will be large, showing their importance.

Original: In scCA, we implement multi-attention and apply them to these compound features.
Revised: In scCA, we implement multi-attention and apply them to these compound features.

Original: We generate several combinations of compound features based on different attention weights.
Revised: We generate several combinations of compound features based on different attention weights.

Original: Then we use the norms of these combinations to serve as the ﬁnal classiﬁcation result.
Revised: Then we use the norms of these combinations to serve as the ﬁnal classiﬁcation result.

Original: [Equation]
Revised: [Equation]

Original: attention weight i =
Revised: attention weight i =

Original: ReLU ( W attention i ∗ sum of feature + b attention i )
Revised: ReLU ( W attention i ∗ sum of feature + b attention i )

Original: [Equation]
Revised: [Equation]

Original: i ∈ [
Revised: i ∈ [

Original: [Equation]
Revised: [Equation]

Original: 2 · · · cell type num ] (4) classification result =
Revised: 2 · · · cell type num ] (4) classification result =

Original: L
Revised: L

Original: [Equation]
Revised: [Equation]

Original: norm ( attention weights
Revised: norm ( attention weights

Original: [Equation]
Revised: [Equation]

Original: compound
Revised: compound

Original: [Equation]
Revised: [Equation]

Original: For implementation, we ﬁrst add all capsules together.
Revised: For implementation, we ﬁrst add all capsules together.

Original: Then we use fully connected layers to generate attention weights.
Revised: Then we use fully connected layers to generate attention weights.

Original: In formula(3), m represents the number of capsules.
Revised: In formula(3), m represents the number of capsules.

Original: In formula(4), cell type num represents the number of cell types in the dataset.
Revised: In formula(4), cell type num represents the number of cell types in the dataset.

Original: We multiply the compound features which are generated by bi-directional LSTM by the attention weights.
Revised: We multiply the compound features which are generated by bi-directional LSTM by the attention weights.

Original: Then we use the L2 norm to get a classiﬁcation result.
Revised: Then we use the L2 norm to get a classiﬁcation result.

Original: Adding attention weights in the model structure lays more emphasis on important compound features.
Revised: Adding attention weights in the model structure lays more emphasis on important compound features.

Original: To provide more illustration for this, We also draw heatmaps of attention weight matrix for every cell.
Revised: To provide more illustration for this, We also draw heatmaps of attention weight matrix for every cell.

Original: We ﬁnd that cells from same cell type share a similar pattern of heatmap, which means certain cell type has their particular attention combinations of compound features.
Revised: We ﬁnd that cells from same cell type share a similar pattern of heatmap, which means certain cell type has their particular attention combinations of compound features.

Original: The ﬁgure above shows 6 cells from dataset GSE70580.
Revised: The ﬁgure above shows 6 cells from dataset GSE70580.

Original: Three cells come from cell type 0 while the other come from cell type 1.
Revised: Three cells come from cell type 0 while the other come from cell type 1.

Original: The deeper the color is in the ﬁgure, the larger the attention weight is.
Revised: The deeper the color is in the ﬁgure, the larger the attention weight is.

Original: We can ﬁnd that attention weights’ distribution vary from one cell type to another.
Revised: We can ﬁnd that attention weights’ distribution vary from one cell type to another.

Original: Different cell types have different attention weight combinations of compound features.
Revised: Different cell types have different attention weight combinations of compound features.

Original: This provide reliable interpretation for our model.
Revised: This provide reliable interpretation for our model.

Original:  E XPERIMENTS 5. I NTRODUCTION OF DATASETS
Revised:  E XPERIMENTS 5. I NTRODUCTION OF DATASETS

Original: Every dataset for single-cell RNA-sequencing consists of a gene expression matrix and cell type labels.
Revised: Every dataset for single-cell RNA-sequencing consists of a gene expression matrix and cell type labels.

Original: Gene expression matrix is the input of our model.
Revised: Gene expression matrix is the input of our model.

Original: Every line in the matrix shows different genes’ expressions in a cell.
Revised: Every line in the matrix shows different genes’ expressions in a cell.

Original: The ﬁgure below demonstrates a sketch heatmap of gene expression matrix.
Revised: The ﬁgure below demonstrates a sketch heatmap of gene expression matrix after data preprocessing mentioned in next section.

Original: The higher the expression level is, the deeper the blue color will be.
Revised: The higher the expression level is, the deeper the blue color will be.

Original: There are many places in the matrix that are shallow, which are the reﬂection of dropouts
Revised: There are many places in the matrix that are shallow, which are the reﬂection of dropouts.

Original: [Table]

Original: We then elaborate the nine datasets we use in experiments, listing out their numbers of cells and genes as well as the numbers of cell types.
Revised: We then elaborate the nine datasets we use in experiments, listing out their numbers of cells and genes as well as the numbers of cell types.

Original: [Table]
Revised: [Table]

Original: 5. P REPROCESSING
Revised: 5. P REPROCESSING

Original: There are  thousands of genes in the dataset.
Revised: There are tens of thousands of genes in the dataset.

Original: Before we train our model, we ﬁrst implement dimension reduction.
Revised: Before we train our model, we ﬁrst implement data preprocessing.

Revised: For every cell in the data, we ﬁrst divide every gene’s expression level by the sum of gene expression levels in the cell.

Original: We calculate the variances of gene’s expression level.
Revised: This makes genes’ expression levels among cells comparable.

Original: Then we choose the largest 2000 genes. We also amplify the data and exert a log transform.
Revised: Then, we multiply the gene expression levels with a constant 100000 and employ a log transform to adjust the expression level in the data into a reasonable range.

Original: 5. P ARAMETER S ETTING
Revised: 5. P ARAMETER S ETTING

Original: For feature extraction part, we generate 8 capsules through fully connected layers.
Revised: For feature extraction part, we generate 8 capsules through fully connected layers.

Original: Each of the capsule has a dimension of 128.
Revised: Each of the capsule has a dimension of 128.

Original: We add  normalization to these capsules and we use a single layer bidirectional LSTM model to generate compound features.
Revised: We add L2 normalization to these capsule vectors and we use a single layer bidirectional LSTM model to generate compound features.

Original: We set the output dimension of LSTM to 16 and dropout of LSTM to 0.3.
Revised: We set the output dimension of LSTM to 16 and dropout of LSTM to 0.3.

Original: A bidirectional LSTM has two output sequences.
Revised: A bidirectional LSTM has two output sequences.

Original: We add them together and get a single sequence of compound features.
Revised: We add them together and get a single sequence of compound features.

Original: To obtain attention weights, we ﬁrst add all capsules together.
Revised: To obtain attention weights, we ﬁrst add all capsules together.

Original: Then we input the sum of capsules into fully connected layers.
Revised: Then we input the sum of capsules into fully connected layers.

Original: The output dimension of these layers is also the number of capsules.
Revised: The output dimension of these layers is also the number of capsules.

Original: The number of attention weights is set to the same as cell type number in the dataset.
Revised: The number of attention weights is set to the same as cell type number in the dataset.

Original: We  multiply thecompound features by these attention weights and get several combinations of compound features. The number of combinations is the number of cell type.
Revised: We then multiply the attention weights to the compound features. After multiplying the attention weights to the compound features, we obtain a matrix whose row number equals to the number of cell types and column number equals to the dimension of compound features.

Original: We use a L2 norm for every combination.
Revised: Then we calculate the L2-norm of every row and apply Softmax function.

Original: Then we get the classiﬁcation result.
Revised: In this way, we obtain a vector whose dimension equals to the number of cell types in the dataset and the sum of the elements in the vector is 1. This vector is the output of scCA. During the training process, cross entropy loss is used as the loss function.

Original: The activation function in our model is ReLU.
Revised: The activation function in our model is ReLU.

Original: We set the dropout to 0.3 and we also add Softmax to the classiﬁcation result.
Revised: We set the dropout to 0.3 and we also add Softmax to the classiﬁcation result.

Original: We split the dataset into training set and test set with the ratio of 4:1.
Revised: We split the dataset into training set and test set with the ratio of 4:1.

Original: The model is trained for 50 epochs with a Pytorch implementation.
Revised: The model is trained for 50 epochs with a Pytorch implementation.

Original: 5. C OMPARISON WITH OTHER METHODS
Revised: 5. C OMPARISON WITH OTHER METHODS

Original: We ﬁrst compare scCA with some machine learning methods, including Support Vector Machine(SVM), Linear Discriminant Analysis(LDA), Decision Tree(DT) and Random Forest(RF).
Revised: We ﬁrst compare scCA with some machine learning methods, including Support Vector Machine(SVM), Linear Discriminant Analysis(LDA), Decision Tree(DT) and Random Forest(RF).

Original: These machine learning methods have no prior knowledge and are implemented using Python.

Original: From the table below, we can ﬁnd that scCA has a robust performance with high accuracy on all datasets.
Revised: From the table below, we can ﬁnd that scCA has a robust performance with high accuracy on all datasets.

Original: It also has some outstanding performances on some of the datasets.
Revised: It also has some outstanding performances on some of the datasets.

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: We then compare scCA with neural network based methods.
Revised: We then compare scCA with neural network based methods.

Original: Here we implement fully-connected network and denoising autoencoder.
Revised: Here we implement fully-connected network and denoising autoencoder.

Original: We use the code provided in scCapsNet.
Revised: We use the code provided in scCapsNet.

Original: From the table below [Table] we can see that among models based on neural networks, scCA still has a good performance.
Revised: From the table above, we can see that among models based on neural networks, scCA still has a good performance.

Original: It outperforms other methods on most of the datasets.
Revised: It outperforms other methods on most of the datasets.

Original: [Table]

Original: In conclusion, through the comparison of our model with other models, scCA shows robustness and high accuracy.
Revised: In conclusion, through the comparison of our model with other models, scCA shows robustness and high accuracy.

Original: On some datasets, scCA has the best performance of all models.
Revised: On some datasets, scCA has the best performance of all models.

Original:  I NFLUENCE OF DIFFERENT PARAMETERS
Revised:  I NFLUENCE OF DIFFERENT PARAMETERS

Original: In our model, the number and dimension of capsules, the dimension of compound features can be adjusted.
Revised: In our model, the number and dimension of capsules, the dimension of compound features can be adjusted.

Original: To test their inﬂuence on scCA’s performance, we carry out experiments under different combinations of parameters.
Revised: To test their inﬂuence on scCA’s performance [Table] we carry out experiments under different combinations of parameters.

Original: [Table]
Revised: [Table]

Original: In table
Revised: In table

Original: [Table]
Revised: [Table]

Original: we set the capsule dimension to 128 and change the number of capsules and the dimension of compound features.
Revised: we set the capsule dimension to 128 and change the number of capsules and the dimension of compound features.

Original: Through the result from the table [Table] we discover that there is no visible improvements no matter we rise the number of capsule or the dimension of compound features
Revised: Through the result from the table, we discover that there is no visible improvements no matter we rise the number of capsule or the dimension of compound features.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: In table
Revised: In table

Original: [Table]
Revised: [Table]

Original: we set the capsule dimension to 64 and change the number of capsules and the dimension of compound features.
Revised: we set the capsule dimension to 64 and change the number of capsules and the dimension of compound features.

Original: Together with table
Revised: Together with table

Original: [Table]
Revised: [Table]

Original: we discover that number of capsules
Revised: we discover that number of capsules

Original: [Table]
Revised: [Table]

Original: the dimension of capsules and dimension of compound features have little inﬂuence on scCA’s performance.
Revised: the dimension of capsules and dimension of compound features have little inﬂuence on scCA’s performance.

Original: Our model performs well under different settings of parameters.
Revised: Our model performs well under different settings of parameters

Revised: [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this article
Revised: In this article

Original: [Table]
Revised: [Table]

Original: we provide a neural network based method scCA for single-cell RNA-seq data cell type classiﬁcation.
Revised: we provide a neural network based method scCA for single-cell RNA-seq data cell type classiﬁcation.

Original: scCA uses capsules to extract features from cells.
Revised: scCA uses capsules to extract features from cells.

Original: Then through a LSTM network it generates compound features which contain information from a set of single features.
Revised: Then through a LSTM network it generates compound features which contain information from a set of single features.

Original: By applying attention weights to these compound features, we lay emphasis on compound features which are more closely related to cell types.
Revised: By applying attention weights to these compound features, we lay emphasis on compound features which are more closely related to cell types.

Original: Deep neural network based, scCA provides reliable interpretations for its result.
Revised: Deep neural network based, scCA provides reliable interpretations for its result.

Original: We calculate the Pearson coefﬁcients among capsules and draw heatmaps of these coefﬁcients.
Revised: We calculate the Pearson coefﬁcients among capsules and draw heatmaps of these coefﬁcients.

Original: We discover that cells from same cell types share a similar pattern of capsules’ relationship while cells from different cell type differ from each other.
Revised: We discover that cells from same cell types share a similar pattern of capsules’ relationship while cells from different cell type differ from each other.

Original: Moreover, we also draw heatmaps of the attention weights.
Revised: Moreover, we also draw heatmaps of the attention weights.

Original: We also ﬁnd out that same cell types emphasize on same compound features.
Revised: We also ﬁnd out that same cell types emphasize on same compound features.

Original: These provide vivid explanations and illustrations for 
Revised: These provide vivid explanations and illustrations for our model.

Revised: We test scCA on nine datasets.

Original: Compared with other machine learning and deep learning methods, scCA shows robustness and has a high classiﬁcation accuracy.
Revised: Compared with other machine learning and deep learning methods, scCA shows robustness and has a high classiﬁcation accuracy.

Original: Furthermore, we adjust the parameters in our model and discover that our model is not sensitive to them.
Revised: Furthermore, we adjust the parameters in our model and discover that our model is not sensitive to them.