A Tutorial on the Mathematical Model of Single Cell Variational Inference

As the large amount of sequencing data accumulated in past decades and it is still accumulating, we need to handle the more and more sequencing data. As the fast development of the computing technologies, we now can handle a large amount of data by a reasonable of time using the neural network based model. This tutorial will introduce the the mathematical model of the single cell variational inference (scVI), which use the variational auto-encoder (building on the neural networks) to learn the distribution of the data to gain insights. It was written for beginners in the simple and intuitive way with many deduction details to encourage more researchers into this field. As the computer technology evolves rapidly, we can tackle more and more complex problem by finding a suitable function taking millions of parameters to model the key part of the problems. The deep neural network(Lecun et al. (2015), Goodfellow et al. (2016) is the top representer of such a function, and it has achieved success in the many fileds, such as natural language processing, image processing, game and so on, now it also gets into the computational biology, e.g. Alphafold(Senior et al. (2020)). This paper will introduce the single cell variational inference model scVI(Romain et al. (2018)), which use the variational auto-encoder equipped with the deep neural networks to tackle the data integration problem and downstream analysis on the scRNA-seq data. The scVI model use the variational auto-encoder to extract information from the gene expression data. The variational auto-encoder consists of the variational encoder and probability decoder, where the encoder will encode the expression data to a continuous hidden low dimensional space in a compact form such that cells from the cell type will close to each other even cells coming from different batches while different cell type separate; the decoder will decode the "code" in the common low dimensional space into the original space, it was designed to separate out the "dropout" effects in the sequencing to get the clean expressed data, which can be used to do imputation and to find differential expressed genes. The "code" in the low dimensional space of the cell can be used to do clustering, annotation, and visualization. The basic idea of the variational auto-encoder is to learn the distribution of the gene expression data by assume that the expression data was generated by two staged processes, the first stage is to sample a code (may be view as the identifier of the cell) from the prior distribution on the low dimensional space, the second stage is to sample a gene expression from a conditional distribution based on the code of the cell. It design a variational decoder to decode the gene expression of a cell into its code, and also a probability decoder to decode the code into the gene expression of the cell. The parameters of the probability distribution of decoder and encoder were output by the deep neural networks. By approximating the log-likelihood by the variational lower lower bound which were calculated from the parameters of the encoder and decoder, we can maximized the variational lower bound to approach the maximum log-likelihood of the observed data, which yields the approximate best probability decoder and encoder. Having the decoder and encoder, we can do the downstream analysis on the data, e.g. clustering, annotation, and visualization, imputation, differential expressed genes and so on. To make an intuitive understand of the scVI model, we will introduce the auto-encoder in Section 1 and variation auto encoder in Section 2. If you are familiar with the auto-ender and variational auto-encoder, please go directly into the Section 3 for the mathematical model of scVI. ar X iv :2 10 1. 00 65 0v 1 [ qbi o. O T ] 3 J an 2 02 1 A TUTORIAL ON THE MATHEMATICAL MODEL OF SINGLE CELL VARIATIONAL INFERENCE 1 Auto-Encoder To understand the scVI model, we should first understand how the variational auto-encoder works. And to make the understanding the variational auto-encoder easier, we first introduce the auto-encoder(Bengio (2009)) which is a similar but simple model. Now, let we think a simple example to get the ideas of auto-encoder. Suppose that the hidden code z ∼ Normal(0,1) and the data  is generated by  = g(z) = [z,2z]. We have  ∼ Normal([0,0], [1,2; 2,4]) (1) formally, where the variance matrix of   =  1 2 2 4  (2) is not invertible, and  is a degenerated normal distribution. Suppose that we see a set of samples of ,